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ABSTRACT 



This study investigates how the structure of the high school 
curriculum influences how far graduates get in the secondary mathematics 
course pipeline, and their level of achievement in that subject by the end of 
high school. The study draws on data from the High School Effectiveness 
Supplement (HSES) of NELS:88, a broad-based longitudinal study of U.S. 
adolescents' experiences and accomplishments in public and private secondary 
schools in and around the 30 largest cities in the U.S. , as well as from the 
students' high school transcripts. Using a two-stage stratified sample of 
3,430 high school students in 184 high schools, the study used the 
Hierarchical Linear Models (HLM) statistical software to estimate school 
effects on individual students' behaviors and achievement. It was found that 
completing high-level mathematics courses is strongly associated with 
students' achievement; however, how schools structure their mathematics 
curricula influences how far their students get in the math course pipeline. 
Students who attend schools where more credits are accumulated in low-level 
mathematics courses make less progress to the more advanced courses. 
Curriculum structure, the types and numbers of mathematics courses offered 
and taken, has both a direct and an indirect effect on students' achievement 
in that subject. Implications of curriculum structure for students' academic 
progress are discussed. (Contains 50 references.) (Author/ASK) 
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Preface 



In this study, the authors investigated how the structure of the high school curriculum 
influences how far graduates get in the secondary mathematics course pipeline and their 
level of achievement in that subject by the end of high school. The study draws on d a t a 
from the High School Effectiveness Supplement (HSES) of NELS:88, abroad-based 
longitudinal study of U.S. adolescents’ experiences and accomplishments in public and 
private secondary schools in and around our nation’s 30 largest cities, as well as from the 
students’ high school transcripts. Data are available from the U.S. Department of 
Education through its National Center for Education Statistics. Using a two-stage 
stratified sample of 3,430 high school students in 184 high schools, the study used the 
Hierarchical Linear Models (HLM) statistical software to estimate school effects on 
individual students’ behaviors and achievement. 

The authors found that completing high-level mathematics courses is strongly associated 
with students’ achievement. However, how schools structure their mathematics curricula 
influences how far their students get in the math course pipeline. Students who attend 
schools where more credits are accumulated in low-level mathematics courses make less 
progress to the more advanced courses. Curriculum structure— the types and numbers of 
mathematics courses offered and taken — has both a direct and an indirect effect on 
students achievement in that subject. Implications of curriculum structure for students’ 
academic progress are discussed. 
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High School Curriculum Structure: Effects on Coursetaking and 
Achievement in Mathematics for High School Graduates 



The publication of A Nation at Risk (National Commission on Excellence in Education 
1983) jolted the public into critical scrutiny of the American high school. How best to 
organize a high school to maximize what its students actually learn increasingly has 
engaged policymakers, educators, and researchers since that time. These examinations 
have found wanting almost all aspects of high schools (for critiques, see Boyer 1983; 
Goodlad 1984; National Association of Secondary School Principals 1996; Powell, 
Farrar, & Cohen 1985; Sizer 1984, 1992). Furthermore, a consistent theme of these 
writings is a call for fundamental change in the structxire of U.S. secondary schools. 

The study described in this report focuses on an important feature of secondary schools: 
the curriculiun. The study investigates how the structure of the high school curriculum 
influences the courses students take, how students’ academic and social background 
characteristics are mapped onto their course choices, and ultimately how secondary 
school curriculiun structiue influences student achievement. The curriculum of high 
schools is organized around subject areas: English, foreign language, mathematics, 
science, social studies, and fine arts. Here we focus on curriculiun and achievement in 
one subject only: mathematics. To maximize the influence of the curriculum on students’ 
academic behavior and performance, we limit our study sample to students who stayed in 
high school for four years, until graduation. 

Background 

Conflicting Goals in the High School Curriculum 

Any high school’s curriculum reflects a formal codification of choices about what 
knowledge is deemed both worthy of transmission to younger generations and within the 
capacity of its students to master. Tensions between worth and appropriateness of 
knowledge embodied in the curriculum leads, in most high schools, to dual goals 
represented within its structure: differentiation and constraint. Balancing these two goals 
represents an attempt to form consensus around a set of common understandings and also 
to address individual differences in students’ abilities and interests (Oakes, Gamoran, & 
Page 1992; Kleibard 1986). Curriculum differentiation reflects the diversification 
approach to this struggle. Here different knowledge is available to different groups of 
students, based on their aptitudes and tastes. This is part of an underlying dynamic that 
attaches distinct purposes and missions to the task of educating students within a single 
high school building. On the other side is a constrained curriculum approach, which 
promotes a belief about the appropriateness of a single set of academic goals for all 
participants. Under this approach, student choices and options are limited. 

The high school curriculum can be constrained through limited opportunity as well as 
through proactive emphasis on moving students into and through a common set of 
experiences. Much of the research about this topic is fimned within the “student choice” 



model: individual students choosing particular courses, and the influence of those choices 
on achievement. However, another research strand (mostly sociological in nature) has 
conceptualized the link between coursetaking and achievement as primarily a school, 
rather than a student, phenomenon. This model assumes that students partake in the 
curriculum of their school to the extent that it is available, and more subtly to the extent 
that they are encouraged to do so (Lantz & Smith 1981; Lee & Ekstrom 1987; Useem 
1991). If the school offers a modest number of courses, largely academic in nature, these 
are the kinds of courses students take (Lee 1993). Thus, a constrained curriculum is 
evidenced both structurally, through the numbers of academic and non-academic options 
provided students, and behaviorally, through the actual variety of choices students make 
in carrying out their courses of study in high school. 

Historical Backgroimd 

The comprehensive high school . Before 1 900, fewer than ten percent of 1 4 to 1 7 year 
olds were enrolled in secondary schools (Oakes 1985). Such schools were elite college- 
preparatory academies, with a narrow focus on a limited range of academic knowledge. 

In the first decades of the twentieth century, enrollment increased dramatically as more of 
this age group attended school and the U.S. population burgeoned through immigration. 
Both immigrant and migrant populations flocked to large cities; this movement was 
accompanied by economic and political shifts to restrict and finally to outlaw child labor 
(Cremin 1988). In response, secondary schools shifted their missions radically. Not all 
students who crossed the thresholds of public high schools plaimed to attend college, or 
even to stay in school for four years. As the types of students they were to educate 
diversified, the schools changed their goals and agendas. 

Debates over what form the public high school curriculum should take polarized around 
two alternatives: (1) a common core of courses meant to be appropriate for all, or (2) a 
diversified set of offerings to accommodate the variety of students. As the debate settled 
in favor of the latter, the comprehensive high school was bom (Conant 1959; Oakes 
1985; Tyack 1974). By providing a wide array of options, “high schools would serve 
democracy by offering usable studies to everyone, rather than dwelling on academic 
abstractions that would interest only a few” (Powell, et al 1985, p.260). An important 
objective of the comprehensive curriculum has always been to keep students in school 
imtil graduation. 

The constrained curriculum . Although similar demographic changes affected almost all 
high schools in the early years of this century, not all of them (especially non-public 
schools) abandoned a commitment to rigorous academic training (Kleibard 1986). 

Because most new immigrants were Catholic, the rapidly expanding Catholic educational 
system felt the need to accommodate its children. The same vigorous debate about the 
purposes and methods of high schools that had occurred among public school educators 
raged among those concerned with Catholic education. Issues of expansion, demographic 
shift, and cumculum focus were central to the debate among educators in both sectors, 
but the debate was resolved quite differently in the two settings. Settling on a rationale 
for a classical curriculum, the Catholic Church argued that developing a student’s ability 



to reason was “necessary in order to grasp fully the established understandings about 
person, society, and God. Although universal secondary education had expanded the base 
of people to be educated, the purpose of education should not change. Practical education 
deviated too far from the central moral aims of schooling” (Bryk, Lee, & Holland 1993, 
p.31). 

Even with rather similar student populations in public and Catholic secondary schools 
(particularly in the cities, where most Catholic schools have always been located), the 
high school curriculum in the two sectors diverged. Curriculum in the two sectors has 
always been based on fundamentally different views about what an appropriate secondary 
education should be. The comprehensive model currently dominates U.S. public high 
schools; the narrow academic curriculum still typifies Catholic schools. There are some 
basic disagreements among educators in the two sectors about whether social and 
intellectual differences among students are best addressed by diversifying instruction and 
content. The response to such diversity rests on basic differences about the ultimate aims 
of secondary education. 

Much of the recent research investigating the structure of the high school curriculum has 
been formulated in cross-sector comparisons. However, one recent study pursued this 
issue only in public high schools (Lee, Croninger, & Smith 1997). The authors concluded 
that a constreiined curriculum in mathematics has positive effects on high school 
achievement in that subject. In that study, “constrained curriculum” was measured in 
several ways: by students taking large numbers of acade m ic courses in mathematics, by a 
high proportion of the school’s student body being in the college preparatory track, and 
by the proportion of the school’s total mathematics offerings that are academic. An 
important aspect of that study was its demonstration of considerable variation in 
curriculum structure in U.S. public schools. That finding undercuts potential critiques 
about the validity of cross-sector research that are based on potential selectivity bias as 
the logical explanation for the prevalence of the constrained curriculum in Catholic high 
schools. 

Cruriculum reform in the 1980s . The Nation at Risk report spmred some movement away 
from the diversified curriculum in public high schools in recent years. Efforts to upgrade 
the curriculum have been a major thrust of educational reform in the last decade, in the 
direction of more students taking more challenging courses (Clune & White 1992; 
McDormell 1988; National Commission on Excellence in Education 1983). Taking 
academic courses has been identified as a primary determinant of achievement (Gamoran 
1987; Jones, et al 1986; Lee & Bryk 1988). This association has provided empirical 
evidence consistent with this policy objective. 

Two enduring educational aims motivate this type of reform of the high school 
curriculum: (1) U.S. students’ performance on standardized tests (excellence), and (2) 
unequal educational outcomes associated with students’ social backgrounds (equity). 
These concerns, and their social and economic implications, have helped shift die 
proportion of the curriculum that is academic back to levels that were common prior to 
the 1930s (Angus & Mirel 1995). There is evidence that such reforms, typically 



manifested by states and districts increasing graduation requirements, have resulted in 
students taking more academic courses. However, these reforms have not had the 
expected result of reducing student coursework in non-academic areas of the curriculum. 
Rather, students have taken /wore courses altogether (Clune & White 1992). 

Unresolved issues regarding curriculum reform . Guided by the dual motivations of 
excellence and equity, reforms of the last decade have attempted to improve the high 
school curriculum and students’ courses of study. However, fundamental normative 
issues underlie decisions about curriculum. For example, those who make decisions about 
curriculum might ask: “What courses should students take in high school?” “Should all 
students take the same courses?” “What courses should be offered and which should be 
dropped from the curriculum?” “What criteria should guide schools’ decisions about 
what courses are offered and students’ decisions about what courses are taken?” “Should 
schools guide students in choosing their course of study in high school, or should they 
leave the choice of courses to students and their families?” Although such questions are 
not addressed directly in this study, we hope that the results will provide empirical 
evidence to inform discussions and decisions about the high school curriculum. 

A Conceptual Model for Studying Curriculum Effects 

Cross-sector comparisons . Our conceptualization of the issue of how high school 
curriculum structure influences student achievement is rooted in research that compares 
Catholic and public schools. From the flurry of recent research comparing the 
effectiveness of schools in the two sectors, a consistent finding concerns the organization 
of the curriculum. One reason why Catholic school students achieve at higher levels than 
their public school counterparts, and why achievement there is also distributed more 
equitably among students of varying backgrounds is the difference in the academic 
organization of high schools in the two sectors (Bryk, et al 1993; Coleman, Hoffer, & 
Kilgore 1982; Lee 1985; Lee & Bryk 1988, 1989). 

Virtually all students in Catholic high schools, regardless of race, social class, aspirations, 
or academic preparation, follow close to the same course of study: a narrow set of mostly 
academic offerings almost all of which are required. The courses that public school 
students take are more differentiated — they are allowed more choices, and their choices 
are more strongly associated with their backgrounds. Recent research, within the 
framework of school restructuring, has added empirical support for the associations 
between a school’s academic organization and both effectiveness and equity (Lee & 

Smith 1995; Lee, Smith, & Croninger 1997).* 

The mathematics curriculum . We decided to focus our study of curriculum and 
achievement on one subject: mathematics. Studies of the effects of tracking and grouping 
on student performance show the largest and most consistent effects in mathematics 
(Gamoran 1987; Lee & Bryk 1988). Mathematics is an area of the curriculum where 
learning is particularly responsive to school experiences (Mumane 1975). In this subject, 
in terms both of content and task expectations, performance measures more closely 
approximate the training that students receive in school than other school subjects. 



Explanations for this match include a more standardized secondary curriculum in 
mathematics than in other subjects, heavy reliance on a modest number of widely used 
texts, and traditional training of mathematics teachers (Romberg 1992). The close 
alignment between teaching and testing suggests mathematics as a fruitful subject in 
which to locate a study of the effect on achievement of course offerings, course 
selections, and other aspects of the structure of the high school curriculum. 

In sum, we targeted this subject for several reasons: (1) its linear sequencing, (2) the 
ability to identify course content by course title (e.g., “Algebra 11” has more specified 
content than “English 11”); (3) the differentiation of the ciuriculum by course level; (4) 
the recognized importance of mathematical skills for college entry and a wide range of 
professions; and (5) the fact that it is learned almost entirely in school and not at home 
(Mtimane 1975). A focus on mathematics allows study of the multi-dimensional nature of 
schools’ influence on individuals’ achievement: course offerings, school policies, course 
sequences, student coursetaking, and students’ performance in school. 

The model . Figure 1 presents the heuristic model that guided our investigation of the 
effects of curriculum structure on learning. The study’s format is hierarchical; students 
are “nested” in high schools. Therefore, the model incorporates constructs of two types: 
those that describe students (their ability, their demographic characteristics, the courses 
they complete, and their achievement) and those that describe schools (their demographic 
composition, structural characteristics such as size and sector, and the structure of their 
mathematics curriculum). 

Figure 1 takes the form of a multilevel path model. Characteristics of both students and 
schools are hypothesized to influence students’ decisions about the mathematics courses 
taken in high school. We hypothesize that coursework, in turn, influences achievement. 

As described below, we operationalize the construct of coursework with the most 
advanced course in mathematics that students complete in high school. The path structure 
of our model means that we may test whether characteristics of students and schools have 
both a direct and an indirect effect on achievement, with the indirect effect mediated by 
the coursework students pursue. 



Figure 1 

Heuristic model for the effects of high-school curriculum structure on mathematics achievement 




Source: U.S. Department of Education, National Center for Education Statistics, High School Effectiveness Study (HSES:90/92). 
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Another feature of the heuristic model shown in figure 1 is that we hypothesize that 
characteristics of schools (in particular the structure of the mathematics curriculum) 
influence not only coursework and achievement but also the social distribution of these 
outcomes. In this model we test whether coursetaking and achievement differ across high 
schools. Beyond differences between schools on those measures, we also investigate 
whether there is significant variation between schools in how coursework and 
achievement are distributed among students within schools according to their social 
characteristics (in this case, family socioeconomic status). As described below, this study 
makes use of a statistical methodology — Hierarchical Linear Models, or HLM — ^that 
capitalizes on the multilevel nature of the data and the hierarchical nature of the 
questions. 

The research questions . Although there are several sets of relationships investigated in 
analyses suggested by figure 1, three major research questions drive this study. All three 
questions are multilevel, and for all three we take into account a substantial set of 
statistical controls (for students and schools). 

Research question 1: Coursetaking. How does the structure of high 
school mathematics curriculum influence the courses students complete 
in high school? In investigating this question, we take into account 
students’ social and academic backgroimds, as well as the composition 
and structural characteristics of the schools they attend. 

Research question 2: Achievement. How does the structure of the 
high school curriculum affect students’ achievement in mathematics at 
the end of high school? The analyses that address this question include 
controls for students’ academic and social backgrounds, their high 
school courses of study, and the demographic and structural 
characteristics of their high schools. 

Research question 3: Equitable distribution of coursetaking and 
achievement. How does curriculum structure influence the relationship 
between students’ socioeconomic status and both the courses they 
complete and their achievement? These analyses also include controls 
for other characteristics of students and schools. 

For all three questions, we hypothesize that a more constrained mathematics curriculum, 
where more students complete the same set of high-level courses in that subject and 
schools offer fewer lower-level courses, is associated with higher levels of academic 
outcomes. That is, we hypothesize that a constrained curriculum results in students 
getting farther in the math course pipeline, in higher levels of achievement in 
mathematics, and in a more equitable social distribution of coursetaking and 
achievement. 



Method 



Data and Sample 

The NELS:88 data . The sample used in this study was drawn from a data supplement to 
the first follow-up of the National Education Longitudinal Study of 1988 (NELS:88), 
sponsored by the National Center for Education Statistics [NCES] (Ingels, et al 1994). 
The purpose of NELS has been to document the educational status and progress of a 
nationally representative sample of U.S. secondary schools and their students. The NELS 
base year (1988) data collection was accomplished with a stratified design structure: 
schools were first sampled, and then a fixed number of students was sampled within each 
school. This resulted in a sample of about 25,000 eighth grade students in 1,035 
American middle grade schools. The NELS design called for biennial data collection, 
with the same students followed into high school, where data were collected on them 
(and their schools) as high school sophomores (1990), seniors (1992), and two years after 
graduation (1994). 

Base year data on NELS:88 students as eighth graders came from several sources: (1) a 
parent survey soliciting information on demographic and academic conditions of the 
home (usually provided by the mother); (2) a broad based survey of students’ attitudes, 
behaviors, and aspirations relevant to education; (3) tests of students’ achievement in 
mathematics, science, reading, and social studies; (4) data from two of their teachers 
(either English or social studies, and either mathematics or science); and (5) inf ormation 
describing schools from principals. Students, teachers, and parents also provided 
descriptive information about their schools. 

The HSES data . The High School Effectiveness Study (HSES) is an independent, 
longitudinal survey executed within the NELS:88 data collection. Its purpose was to 
augment the original samples of students within high schools from the earlier NELS 
study, in order to make possible school effects studies (such as the one described here). 
Although it is a separate study, HSES borrows heavily from NELS:88, as it includes 
many of the same students and schools and follows a similar design. The data are selected 
through a complex, two-stage, stratified sample design, and NCES provides school 
weights to compensate for oversampling of certain types of schools (especially private 
schools and schools enrolling high proportions of minority students). Results of analyses 
using the HSES data set are therefore able to be generalized to the population of tenth to 
twelfth graders in urban and suburban secondary schools in the 30 largest metropolitan 
statistical areas (MSAs). 

At the first step of the sampling, all high schools (urban or suburban schools in the 30 
largest MSAs) enrolling one or more NELS:88 students at the first follow-up (1990) were 
selected (724 schools). A stratified sample of 276 schools from this pool was drawn, 
resulting in a final baseline sample of 247. The High School Effectiveness Study Data 
File User’s Manual (Scott, et al 1996) provides complete details on the sampling 
strategies used for selecting schools and students. All NELS:88 first follow-up students 



attending the 247 schools in the HSES school sample were included in the HSES student 
sample. 

The student sample was then augmented, using a sampling algorithm that took into 
accoimt tenth grade enrollment of the HSES school. The original aim of the HSES was to 
produce a within-school cluster size of approximately 30 students. This has resulted in 
two distinct groups of students in each school: (1) those who were part of the original 
NELS sampling design, and (2) those who were added in the augmentation, having 
entered the sample only as tenth graders. As a result, the second group of students does 
not have data as eighth graders (i.e., before they entered high school). Thus, the HSES 
base year is 1 990, when students were high school sophomores. Base year data for HSES 
come from the same sources as in NELS:88: students, parents, teachers, and 
administrators. 

The analytic sample . Our investigation drew upon information about both students and 
schools, focusing on students who stay in school (and in the same school) through Grade 
12. That is, we focused on high school graduates — ^those students with maximum 
exposure to their schools. We thus selected our sample in a two-stage fashion based on 
the availability of (1) student-level and (2) school-level data. Our analytic sample for this 
study was drawn from the available HSES sample of 7,642 students and 247 schools.^ 
The analytic sample includes only those students with student questionnaires 
(S290QPFL) and complete tenth and twelfth grade mathematics test scores (S12XMIRR, 
S22XMIRR).^ From the original sample, 5,449 students (and 238 schools) had tenth and 
twelfth grade questionnaire data available. Of those students with complete questionnaire 
data, 4,232 (and 230 schools) also had mathematics test scores both in 1990 and in 1992.^ 

Because our study focused on high school curriculum and student coursetaking, it was 
crucial that sampled students had complete transcript information available for Grades 9 
through 12 (S2RTR09, S2RTR10, S2RTR1 1, S2RTR12). Employing this filter fiirther 
reduced the sample to 3,741 students (and 213 schools). We selected only students with 
complete information on race (SI RACE), gender (SI SEX), and socioeconomic status, or 
SES (S2SES1), as we intended to include these important demographic characteristics in 
our multivariate models. These demographic selections reduced the sample to 3,724 
students (and 213 schools). 

After selecting students based on available student-level information, we turned our 
attention to key school characteristics. We first selected students from schools with 
administrative questionnaire data available at the two waves (SIADMFLG, 

S2ADMFLG). This selection reduced our sample to 3,63 1 students and 207 schools. We 
chose to omit students in private schools that were neither Catholic nor members of the 
National Association of Independent Schools (NAIS),^ because these “other” private 
schools were a non-homogeneous group with diverse purposes. Therefore, our school 
sample includes only public. Catholic, and NAIS schools (G10CTRL2). This selection 
resulted in a sample of 3,462 students and 198 schools. 



Although not absolutely necessary for the HLM estimation procedure, we eliminated 
students in schools with less than five sampled students in them, a procedure we followed 
in other studies using NELS data (Lee & Smith 1995, 1997; Lee, Smith, & Croninger 
1997).^ This selection resulted in a final sample of 3,430 students in 184 schools. Thus, 
our final analytic sample represents 45 percent of the originally sampled HSES students 
and nearly 75 percent of the originally sampled HSES schools. 

We investigated differences between the analytic sample of included students and schools 
and those students and schools eliminated from the HSES sample, the results of which 
are presented in appendix A. Although we recognize that the reductions in school and 
(especially) student sample sizes are substantial, we believe that the selection criteria 
were necessary in light of the purpose of our study. We note that the major loss of cases 
was a large number students who were nominally included in the sample due to HSES 
selection criteria, but who did not have questionnaire or test information, and thus were 
effectively “absent” from any useful analysis. 

Weights . As Avith other longitudinal data collected under the sponsorship of NCES, 
analyses using the HSES data require the use of design weights to compensate for a 
stratified sampling strategy, unequal probabilities of selection, and to adjust for the 
effects of nonresponse. Both school-level and student-level weights are available on the 
HSES data file, and users must think carefully about selecting the appropriate 
combination of weights. 

At the time we undertook this study, there were questions about the validity of the 
student-level weights included on the HSES file. For that reason, we chose not to weight 
our analyses at the student level for either descriptive or multivariate analyses. We did, 
however, employ the school level weight constructed Avith the “Qian-Frankel” probability 
model. We used the Qian-Frankel weight both for descriptive information on the schools 
and for the multivariate models. Appendix B includes an elaboration of the discussion 
about weights, the rationale for our choices, and the implications of those choices for our 
results. NCES Avill soon make available revised student-level weights with the HSES data 
files. 

Measures 



Outcomes . We employed two dependent variables in this study, mathematics 
achievement at the end of hi^ school (twelfth grade) and highest mathematics course 
completed. Mathematics achievement was drawn from the twelfth grade NELS 
mathematics test. Unlike previous studies, our measure of mathematics coursetaking, 
drawn from student transcript data, represents how far students proceed through the 
mathematics pipeline (i.e., ranging from 1 = no mathematics through 8 = calculus), rather 
than the number of years they elect mathematics. This difference distinguishes our study 
from most other research on curriculum effects. 

Appendix C provides details of the construction of all variables employed in this study. 



Student background characteristics . Because one focus of this study is to investigate how 
socioeconomic status influences the outcomes, we included a measure of students’ SES. 
We also drew on Oakes (1990) for our choice of other independent variables to use as 
covariates. Our covariates on students include: gender, minority status, and prior math 
achievement. The HSES sample only includes eighth grade test scores for the original 
NELS:88 core sample. This posed a serious measurement problem for this study, in that 
two-thirds of the analytic sample had no measure of student achievement and/or ability 
prior to high school (those students who “joined” the HSES sample at Grade 10). For this 
analysis we employed students’ mathematics grade point average (GPA) in the ninth 
grade, taken from student transcript data, as a proxy for prior achievement. 

School composition and structure . Curriculum structure is related to other characteristics 
of schools: school size is strongly related to the number of courses that a school can offer 
(Bryk, et al 1993; Lee 8c Smith 1995; Monk 8c Haller 1993), while average SES, minority 
enrollment, and school achievement levels similarly influence course offerings by 
affecting student interest and demand (Lee, Croninger, 8c Smith 1997). Therefore, we 
included several measures of school composition and structure as covariates: sector (i.e., 
public. Catholic, NAIS), school average social class, school size, minority concentration, 
proportion of low-achieving students, the average ninth grade mathematics GPA, and the 
variability of ninth grade mathematics GPA in each school. 

Mathematics curriculum structure . The mathematics curriculum structure of American 
high schools, and its effects on achievement, is the primary focus of this study. Therefore, 
we included multiple measures to reflect curriculum structure. Two measures captured 
students’ average progress through the mathematics curriculum: (1) the average of the 
highest course completed in each school, and (2) the variability of mathematics 
coursetaking in each school. The first, the school mean, estimates overall progress 
through the pipeline, or how far, on average, students go. The second measure — ^the 
within-school standard deviation [SD] — is an estimate of the imiformity of that 
progression, or how much disparity in coursetaking occurs. We designate schools with 
high average mathematics progress and low variability in mathematics coursetaking as 
“effective.” 

Other variables represented the availability and election of courses below algebra: (3) the 
number of mathematics courses offered below algebra in each school; (4) the average 
number of mathematics credits completed below algebra in each school; and (5) the 
variability (the within-school SD) in the number of mathematics credits completed below 
algebra in each school. The first measure captured the breadth of the low end of the 
mathematics offerings, the initial stages of the high school mathematics pipeline. The 
larger the number of low-level offerings, the greater the potential for certain students to 
remain at these early stages. The second two measures captured the actual coursetaking 
behavior in schools rather than the course offerings. Because students caimot elect what 
is not offered, another variable we employed focused on the high end of the mathematics 
curriculum: (6) whether or not calculus was offered in each school. 



Analysis 



Descriptive information . Our investigation proceeded from descriptive analyses of 
students and schools to multivariate causal models. The structural focus of all analyses 
was on the high school mathematics curriculum and students’ progress through it. We 
began by investigating observed differences among students, differentiated by groupings 
that characterize their progress through the mathematics curriculum. Student 
demographics and achievement were compared for three such groups: (1) those whose 
total high school mathematics course work was below the level of algebra [7.9 percent]; 
(2) those whose total high school mathematics coursework included or exceeded algebra, 
but was below the level of pre-calculus [57.4 percent]; and (3) those whose total high 
school mathematics coursework included pre-calculus or calculus [34.7 percent]. 

In addition to the large variation in mathematics progress between individual students, 
there is substantial variation in average student progress between schools. Consequently, 
we followed a similar strategy for describing the school sample. School demographics 
and curriculum structure were compared for three groups of schools, also differentiated 
by student progress through the mathematics curriculum: (1) those with low (more than 
one SD below the mean) average mathematics coursetaking [16.3 percent]; (2) those with 
middle (within one SD of the mean) average mathematics coursetaking [66.9 percent]; 
and (3) those with high (more than one SD above the mean) average mathematics 
coursetaking [16.8 percent]. 

A hierarchical approach . Since our research questions focused on estimating how the 
structure of the high school mathematics curriculum influenced how far students moved 
through it and their achievement in that subject, we used a method designed for such 
situations — ^Hierarchical Linear Modeling [HLM] (Bryk & Raudenbush 1992; Bryk, 
Raudenbush, & Congdon 1994). In an early school effects study using HLM, Lee and 
Bryk (1989) provided some methodological details within their application. HLM has 
also been used in other studies with designs similar to the study described here (e.g., Lee, 
Croninger, & Smith 1997; Lee & Smith 1993, 1995, 1996; Lee, Smith, & Croninger 
1997). Readers interested in more details about HLM are referred to the studies cited in 
this paragraph, or to Appendix D, where we provide a brief overview of HLM as it is 
applied in this study. 

The multivariate models in this study have a two-level hierarchical structure, with 
studente nested in schools. There are three steps to a multilevel analysis of this type, 
which is often called a “school effects” study. In Step 1 , the variance in the outcomes is 
partitioned into its within- and between-school components. It is only that proportion of 
the total variance that lies systematically between schools that may be modeled as a 
function of school factors. Step 2 involves the construction of a within-school (or Level- 
1 ) model. Here the outcome variable is modeled, separately in each school, as a function 
of the academic and social characteristics of the sampled students in that school. Step 3, 
the between-school (Level-2) model, investigates characteristics of schools that are 
related to the outcomes, which are school-level averages adjusted for the types of 
students in each school. It is Step 3 of an HLM model where school effects are identified. 



The school effects model explored in this study . To pursue our research questions, we 
explored two different multilevel, multivariate models related to the mathematics 
curriculum and students’ progress through the mathematics pipeline. The first model, 
which addresses Research Question 1 , estimated the impact of the high school 
mathematics curriculum on students’ progress through Ae mathematics pipeline. The 
dependent variable for this HLM model was the highest mathematics course completed. 
The Level- 1 model here included controls for students’ gender, minority status, SES, and 
ninth grade GPA in mathematics. The Level-2 HLM model for this outcome included 
school controls for composition and sector (e.g., average SES, proportion of low- 
achieving students, sector, size), and also the several variables that we have used to 
measure high school curriculum structure (e.g., number of courses the school offers 
below algebra, availability of calculus, average number of low-level mathematics credits 
completed). 

The second set of HLM models, intended to explore Research Question 2, estimated the 
impact of the high school mathematics curriculum on students’ twelfth-grade 
mathematics achievement. Student- and school-level controls were identical in the first 
and second models, except for the inclusion of students’ individual mathematics progress 
(as an additional student-level predictor) and the school’s average mathematics progress 
and variability in progress included as additional school-level predictors of achievement. 

Level-2 predictors, which describe schools, were used to model outcomes that 
characterize both effectiveness and equity in U.S. secondary schools. The effectiveness of 
the school in the two models was investigated by estimating effects on the intercepts 
from the within-school models, which were represented by the adjusted school-level 
average of the highest mathematics course taken and mathematics achievement. Although 
equity issues in educational research may also focus on gender or minority status, in this 
study we restricted our equity focus to an investigation of the impact of the school 
mathematics curriculum on the social distribution of coursetaking and achievement. 
Specifically, we modeled the social class effects (the student-level SES slopes) on our 
two outcomes as functions of the school curriculum (slopes-as-outcomes). We 
characterize schools with an equitable social distribution of learning as having reduced 
SES effects on student pipeline progress and achievement. 

Centering of independent variables in HLM . In all quantitative research, decisions about 
scaling and centering variables require careful consideration in order to interpret results 
appropriately. That is, researchers should ask themselves, “What does a value of zero on 
each variable actually mean?” This is especially important for slopes and intercepts of 
Level- 1 HLM models for school effects studies, as these become outcomes in between- 
school investigations (Bryk & Raudenbush [1992] provide a technical discussion of 
centering in HLM). Decisions about where to center variables at Level 1 dete rmin e the 
exact nature of the outcome under investigation. The question each researcher needs to 
keep in mind is this: “What should the value of the within-school intercept represent?” 



In multivariate models for this study, the continuous independent variables in our within- 
school models (SES and ninth grade math GPA) were first transformed into z-scores on 
our sample (mean [M] = 0, SD = 1). The other Level- 1 predictors (minority status and 
gender) were dummy-coded. Further, all within-school predictors for our analyses (SES, 
gender, minority status, and math grades) were centered around their respective school 
means. This decision allowed us to model the distribution of adjusted school means (the 
intercepts) and school SES slopes in a meaningful way.’ Because they do not serve as 
outcomes in HLM models, decisions about where to center Level-2 independent variables 
involve fewer techmcal concerns. Decisions here have substantive implications, however, 
because they influence interpretation of the school effects. Centering independent 
variables at the within-school level determines what is modeled; centering at the 
between-school level determines what the effects mean. 

In our multivariate models, all continuous, Level-2 independent variables were 
normalized (i.e., SD = 1). The continuous measures of school social and academic 
composition (average SES, proportion of low-achieving students, minority concentration) 
were also transformed into z-scores (M = 0, SD = 1). However, the continuous 
curriculum structure measures (number of courses offered below algebra, average 
number of credits below algebra, variability in credits below algebra) were not 
transformed into z-scores, because in these cases a value of zero is “true.” That is, a zero 
on these variables represents no courses offered below algebra or no credits completed 
below algebra. Of course, for dummy-coded Level-2 independent variables (e.g., whether 
the school offers calculus), zero also has substantive meaning. 

Results 

Who Progresses Through the Mathematics Pipeline? Descriptive Results 

Highest mathematics course completed . The highest mathematics course completed in 
high school is not only a record of students’ academic success, but it also serves as a filter 
for subsequent educational and vocational opportunities available to them (Hyde, 
Fennema, & Lamon 1990). Figure 2 displays the distribution of students’ mathematics 
progress. Its negative skew indicates that few students who stay in high school until 
gradtiation remain at the low end. By the time they finish high school, only 8 percent of 
seniors in this sample had dropped out of the pipeline prior to completing a traditional 
Algebra I course (levels 1 , 2, and 3 on this measure), the critical prerequisite for higher- 
level academic mathematics courses in high school. Even though there are small 
proportions of students at the very low end of the pipeline, over 50 percent of gradtiates 
stopped midway through the pipeline (levels 4, 5, and 6, including Algebra I, II, plane 
and analytic geometry). Over a third of seniors completed advanced mathematics courses 
(levels 7 and 8, pre-calculus and calculus), the critical prerequisites for college 
mathematics and science courses. 



% of Students for Whom Each Course is Their Highest 



Figure 2 

Highest mathematics course completed for high school graduates 
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Note; This is an eight-level variable constructed from NELS/HSES student transcript data (student course credits in 
mathematics). The scale was created by selecting the highest level of mathematics course each student 
completed with a passing grade during his/her four years in high school. 




KEY 

1 No Mathematics 

2 Non-Academic 

3 Low Academic 

4 Middle Academic I 

5 Middle Academic II 

6 Advanced Academic I 

7 Adv. Acad. II (Pre-Calc.) 
$ Adv. Acad. Ill (Calc.) 



Source: U.S. Department of Education, National Center for Education Statistics, High School Effectiveness Study (HSES:90/92). 
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Differences between students . Which types of students remain at the low end of the 

pipeline? Which students proceed through the mathematics curriculum into the advanced 

mathematics courses? Virtually all research demonstrates that mathematics coursetaking, 

measured in years, is strongly related to students’ background characteristics (see reviews • 

by Hyde, et al 1 990 and Oakes 1 990). We also explored the relationship between student 

characteristics and mathematics coursetaking. We remind readers that Ae variable we use 

does not measure years of coursework, but rather progress through the mathematics 

course pipeline. Table 1 displays the characteristics of high school graduates who attain 

three levels of pipeline progress: (1) “low”; below algebra, (2) “middle”: algebra but € 

below pre-calculus, and (3) “high”: pre-calculus or calculus. 

Table 1.— Group means of variables describing high school graduates, differentiated 
by the highest mathematics course completed 

(n = 3,430 students in 1 84 schools) ® 



Variable 


Below algebra 
(n-271) 
Mean 


Algebra but below pre- 
calculus (n = 1,969) 
Mean 


Pre-calculus or calculus 
(n - 1,190) 

Mean 


Percent Female 


44* ‘ 


51 


44* 


Percent Minority 


42 


37 


17* 


Black, Non-Hispanic 


18 


17 


8* 


Hispanic 


21 


19 


9* 


Asian 


3* 


8 


17* 


Social Class (SES)^ 


-.26* 

(.63)' 


.14 

(.78) 


.70* 

(.71) 


Mathematics GPA, 


1.66* 


2.21 


3.04* 


Grade 9 


(.90) 


(.94) 


(.84) 


Mathematics Achievement, 


30.71* 


46.55 


64.44* 


Grade 12 


(8.41) 


(12.40) 


(9.53) 


Highest Mathematics 


2.48* 


5.01 


7.55* 


Course Completed 


(.56) 


(.79) 


(.50) 



1. For all variables, we tested two contrasts with one-way analysis of variance: Below Algebra (column 1) 
vs. Algebra but below Pre-Calculus (column 2), and Algebra but below Pre-Calculus (column 2) vs. Pre- 
Calculus or Calculus (column 3). Significance levels of the first contrast are included on means in column 

1. and significance levels of the second contrast are included on means in column 3. 

2. Social class is a z-score variable standardized on the full HSES sample. H 

3. SDs of continuous variables are in parentheses. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, High School 
Effectiveness Study (HSES.90/92) . 
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Gender differences in mathematics coursetaking are mixed. Although fewer girls 
complete pre-calculus or calculus than boys (44 percent of the students in this group are 
females), girls are also less likely to end their mathematics coursetaking with non- 
academic or low-level courses (i.e., below Algebra I). More girls than boys stop taking 
math courses midway through the pipeline, whereas boys typically either stop early or 
make it to the end of the pipeline {t = 2.42, t = 4.04, respectively}. 

Disparities by social class and race/ethnicity are large, and they follow a common pattern. 
Seventeen percent of the students who reach the end of the mathematics pipeline are 
minority students, whereas nearly half (42 percent) of those at low levels are minority {t 
= 8.18}. This is the pattern for black and Hispanic students {t = 6.69, 6 = 5.01, 
respectively}; Asian students do not follow this pattern: although 17 percent of students 
taking pre-calculus or calculus are Asian, 3 percent of those who end with the low-level 
mathematics courses are Asian {t = 7.29}. Social class (SES) is also strongly related to 
pipeline progress: high-SES students move further. Students in the low group have much 
lower average SES than those in the middle group (effect size [ES] = .40 for the group 
difference, t = 8.24).* The largest difference is between students who dropped out of Ae 
pipeline early and those who proceeded to the end (ES = .96, t = 19.22). 

Students’ academic status and progress, measured by mathematics grades in ninth grade 
and achievement near the end of twelfth grade, are strongly related to their progression 
through the pipeline. Students who completed higher-level mathematics courses received 
substantially higher mathematics grades in their first year of high school (GPA = 3.04) 
and displayed less variability in GPA (SD = .84) than their counterparts who did not take 
algebra (GPA = 1.66, SD = .90, t = 22.24 and F = 16.91). Further, Aese students 
exhibited substantially higher mathematics achievement in their final year of high school 
(64.44 vs. 30.71, ES = 3.54 SD, t = 44.75). The SES and performance of students 
remaining midway through the pipeline is more variable than the groups at either extreme 
{F = 15.84 and F = 99.07, respectively}. 

School Factors Associated With Greater Student Progress in the Mathematics 
Curriculum: Descriptive Results 

High- vs. low-progress schools . Although the distribution of students’ progress through 
the mathematics pipeline is negatively skewed, the distribution of school average 
progress is close to normally distributed (see figure 3). Across all schools, the average 
progress ranges from a low near three (a school where the average progress leaves 
students below algebra) to a high of eight (a school where _all_ students reach calculus).^ 
We characterize high-progress schools as those where average student progress through 
the mathematics pipeline is more than one SD above the mean of this variable across 
schools; low-progress schools are those in which average student progress is more than 
one SD below the mean. Because this variable is close to normally distributed, a thir d of 
all schools fall in either of these two categories, with as many schools of high as of low 
progress. 



% of Schools at Each Level 



Figure 3 

School average of highest mathematics course completed for high school graduates 
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1 No Mathematics 

2 Non-Academic 

3 Low Academic 

4 Middle Academic I 

5 Middle Academic II 

6 Advanced Academic 1 

7 Adv. Acad. 11 (Pre-Calc.) 

8 Adv. Acad. Ill (Calc.) 



Note: This is an eight-level variable constructed from NELS/HSES student transcript data (student course credits in mathematics). The 
scale was created by selecting the highest level of mathematics course each student completed with a passing grade during his/her 
four years in high school. The school average was computed and rounded to the nearest .5. 

Source: U.S. Department of Education, National Center for Education Statistics, High School Effectiveness Study (HSES:90/92). 
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What distinguishes schools with different levels of average progress in the mathematics 
course pipeline? Table 2 presents information on school composition and structure, 
mathematics curriculum structure, and average school achievement for three groups of 
schools: (1) schools of low average progress [ranging from 3.0 to 4.43; 15.8 percent of 
the schools]; (2) schools of middle average progress [ranging from 4.43 to 6.5; 66.8 
percent of Ihe schools]; (3) schools of high average progress [ranging from 6.5 to 8.0; 
17.4 percent of the sample]. 

Differences between schools . School composition and structure. Descriptive differences 
between the school groups in table 2 appear to be larger than those between the groups of 
students shown in table 1 . Consistent with individual predictors of progress, high average 
levels of pipeline progress are associated with low-minority and high-SES enrollments, 
and fewer low-achieving students. Conversely, three fourths of the low-progress schools 
have high minority enrollments (40 percent or more minority students), and, on average, 
nearly half of the students in low-progress schools earned a C or below in ninth grade 
mathematics. There are also differences by school sector: all of the low-progress schools 
are public schools; 14 percent of the high-progress schools are public schools {t = 6.48}. 
These findings are consistent with other research that has shown large differences in 
academic coursetaking between students in public and Catholic schools (e.g., Bryk, et al 
1993; Lee & Bryk 1988). High-progress schools are also substantially smaller, on 
average {t = 5.1 1 }, a structural feature associated with school sector. 



Table 2.-Group means of variables describing high schools, differentiated by the 
school average of students’ highest mathematics course completed 

(n = 184 schools) 



Variable 


Low average coursetaking 
(n = 30) 


Middle average coursetaking 
(n = 123) 


High average coursetaking 
(n = 31) 


A. School composition 








Percent high minority 
schools 


75*' 


23 


4* 


Percent of low-achieving 
students 


47* 


29 


26 


Average SES 


-.26* 


.23 


.83* 


B. School structure 


(.36)' 


(.40) 


(.22) 


Percent Catholic schools 


0 


13 


55* 


Percent NAIS schools 


0 


2 


31* 


School size 


1426 


1300 


649* 


(785) 

C. Structure of mathematics curriculum 


(748) 


(425) 


Percent offering calculus 


45* 


83 


100* 


Mean, number of courses 


4.63 


4.14 


2.11* 


offered below algebra 


(.82) 


(1.55) 


(1.92) 


Mean, credits taken 


1.21* 


68 


.09* 


below algebra 


(.44) 


(.40) 


(.11) 


SD, credits taken below 
algebra 

D. Student Achievement 


.96* 


.82 


.24* 


Mean, math GPA, grade 


1.96* 


2.38 


2.64* 


9 


(.29) 


(.40) 


(.30) 


SD, math GPA, grade 9 


.93 


.96 


.98 


Mean, math achievement. 


40.42* 


50.46 


58.41* 


grade 12 


(5.15) 


(6.59) 


(5.33) 


SD, math achievement, 
grade 12 


13.13* 


12.03 


12.15 


Mean, highest math 


4.08* 


5.42 


7.21* 


course completed 


(.35) 


(.54) 


(.57) 


SD, highest math course 
completed 


1.35 


1.39 


.81* 


p<.05 









1 . For all variables, we tested two contrasts with one-way analysis of variance: Low Average Coursetaking 
(column 1) vs. Middle Average Coursetaking (column 2), and Middle Average Coursetaking (column 2) vs. 
High Average Coursetaking (column 3). Significance levels of the first contrasts are included on the means 
in column 1, and significance levels of the second contrast are included on the means in column 3. 

2. SDs of continuous variables are in parentheses. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, High School 
Effectiveness Study (HSES:90/92). 
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Curriculum structure . It is expected that average coursetaking reflects curriculum 
structure. All of the high-progress schools offer calculus (a necessity for students to elect 
it) but less than half of the low-progress schools offer this course. Although not offering 
calculus prevents students from taking the course, it may also be a reasonable omission 
for schools in which there is little demand for the course. 

Low- and middle-progress schools offer nearly twice as many math courses below 
algebra as high-progress schools. Furthermore, low-progress schools more consistently 
offer a larger number of such courses (the SD of the means = .82), while high-progress 
schools — ^those that offer fewer such courses on average — display much more variability 
(SD of the means = 1 .92, F = 1 7.00). Although low- and middle-progress schools offer 
similar numbers of these courses, students in low-progress schools complete more low- 
level coursework than students in middle-progress schools (average number of credits per 
student: 1.2 vs. 0.7, t = 6.73).'° Almost no students in high-progress schools complete 
coursework below the level of algebra (average number of credits per student: .09). 

Students in low-progress schools display substantially more variability in low-level 
coursetaking than students in high-progress schools (F = 10.63}. In low-progress 
schools, many students take additional credits below algebra, while other students in the 
same school are taking no credits below algebra (these bimodal distributions result in the 
larger average SD for low-progress schools [.96]). High-progress schools display the 
least variability in low-level mathematics covirsetaking: in the typical high-progress 
school, very few per-pupil credits below algebra are completed (M = .09), and high- 
progress schools consistently display this pattern (SD of the means = .11). Furthermore, 
students within the high-progress schools are more similar in their coursetaking behavior 
(average SD = .24, t = 1 1 .8)." 

School achievement . The achievement-related antecedents and results of high progress in 
schools are clear and consistent. On average, students in high-progress schools did better 
in their ninth grade math courses (GPA = 2.6, or a B-) than students in low-progress 
schools (GPA = 2.0, or a C, t = 7.37), although all schools (low-, middle-, and high- 
progress schools) display similar within-school variability in grades. Twelfth-grade 
mathematics achievement is over 40 percent higher in high- than low-progress schools 
(58.4 vs. 40.4; a difference of ES = 1.09 SD, t = 1 1.59). 

Summary of descriptive differences . On virtually every variable we considered in ovn 
multivariate models, both between students and between schools, there were statistically 
significant differences related to students’ progress through the mathematics pipeline. 
Group differences among students were associated with social background (SES, 
race/ethnicity) and academic performance at both the beginning and end of high school. 
There were also differences in average pipeline progress by school sector, by school size, 
and by the structure of the high school curriculum. Although these differences suggest 
curriculum structure effects on student achievement, it is important to remember that the 
mean differences displayed in tables 1 and 2 are descriptive — ^they do not take other 
characteristics of students and schools into account. We know that many of the variables 
we consider here are correlated. Thus, we investigate these questions within a 



multivariate format. As we discussed, it is also appropriate to consider these questions 
within a multilevel format. We move to these analyses now. 

Multivariate and Multilevel Analyses 

Unconditional HLM models . Table 3 displays the psychometric properties of the two 
outcome variables in this study: the highest mathematics course completed and 
mathematics achievement at twelfth grade. These results were computed as fully- 
unconditional HLM models of the two outcomes. Two statistics are especially useful 
here. First, we see that each of these outcomes demonstrated moderately high (and quite 
similar) lambda reliability (.770 for coursework; .719 for mathematics achievement). As 
school effects may be estimated only on the proportion of variability in the outcome that 
is systematically between schools, the intraclass correlation (ICC) is also informative. 

The estimated ICCs here were relatively high. Over two-fifths of the total variance in the 
measure of highest course completed (41.3 percent) lay between schools. Although not 
quite so high, more than one-third (35.2 percent) of the total variance in twelfth-grade 
mathematics achievement was between schools. These ICCs are somewhat higher than 
those in similar studies investigating the effects of the high school curriculum on student 
achievement (Lee &, Bryk 1989; Lee &, Smith 1995; Lee, Croninger, &. Smith 1997). The 
relatively hi^ ICCs of these outcomes, combined with their moderately high reliabilities, 
indicate the likely presence of school effects. 

Table 3.--HLM psychometric properties of student outcomes: Highest mathematics 
course completed and mathematics achievement in grade 12^ 

(n = 3,430 students in 1 84 schools) 



Model 1 Model 2 

Highest Mathematics Course Completed Mathematics Achievement 



Pooled Within-School Variance 
(Sigma-Squared) 


.592 


.685 


Between-School Variance (Tau) 


.321 


.267 


Within-School SD^ 


.769 


.828 


Between-School SD ^ 


.567 


.753 


Reliability (Lambda) 


.770 


.719 


Intraclass Correlation, Unadjusted (ICC)^ 


.352 


.280 


Intraclass Correlation, Adjusted (ICC)^ 


.413 


.352 



1. Outcome variables are in a z-score metric (M=0, SD=1) for this and other HLM analyses. 

2. The within-school SD is the square root of sigma-squared; the between-school SD is the square root of 
Tau. 

3. Unadjusted ICC = Tau /(Tau + Sigma; Adjusted ICC = Tau /(Tau)+(Sigma * Lambda). 

SOURCE: U.S. Department of Education, National Center for Education Statistics, High School 
Effectiveness Study (HSES:90/92). 



Within-school HLM models . Table 4 displays three within-school (Level 1) HLM 
models. Each model is similar, in that it has a similar set of independent variables. 
Because all variables included in these models were either z-scored or dummy-coded, the 
beta coefficients shown in table 4 are in an effect-size (SD) metric (see technical note 8). 
Using effect sizes allows us to interpret effects both substantively and statistically, 
(nominal significance levels are indicated by p- values). Model 1 of table 4 displays 
results for our within-school analysis of the highest mathematics course completed; 
Model 2 displays similar results for mathematics achievement. What distinguishes 
Models 2A and 2B is that the highest mathematics course completed is included in the 
within-school HLM in Model 2B but not in Model 2A. In these models, the intercept and 
SES slopes become outcomes in our between-school HLM models to follow. 



Table 4.—Within-school HLM models of highest mathematics course completed and 
mathematics achievement in grade 12 

(n = 3,430 students in 184 schools) 



Variable 


Model 1 

Highest mathematics 
course completed 

P Coefficient 


Model 2A 

Mathematics achievement 
without coursetaking 

P Coefficient 


Model 2B 

Mathematics achievement 
with coursetaking 

P Coefficient 


Intercept(Po) 


-.066 


-.057 


-.064 


Social Class (SESXPi) 


.194* 


.166* 


.034 


Gender (Female)(P 2 ) 


-.055* 


-.179* 


-.141* 


Minority Status(P 3 ) 


-.046 


-.333* 


-.305* 


Mathematics GPA, (P 4 ) 


.310* 


.338* 


.131* 


Grade 9 


Highest Mathematics (P 5 ) 






.658* 


Course Completed 


Parameter 


The Chi-Square Table 

Estimated parameter Degrees of variance(Tau) 


Chi-square freedom 


Highest math course 


Intercept 


.332 


183 


2855* 


SES slope 


.027 


183 


231* 


Mathematics achievement 
(Without coursetaking) 


Intercept 


.290 


183 


2551* 


SES slope 


.034 


183 


215* 


(With coursetaking) 


Intercept 


.326 


183 


4233* 


SES Slope 


.031 


183 


212 


*p<.05 









SOURCE: U.S. Department of Education, National Center for Education Statistics, High School 
Effectiveness Study (HSES.90/92) . 
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Model 1 . Both SES and ninth grade mathematics GPA are positively and significantly 
related to mathematics course pipeline progress, although the magnitudes of effects are 
small for SES (I> = .194, t = 7.85) and moderate for GPA (I> = .310, t = 23.36). Gender 
has a significant effect that is quite small (I> = -.055, t = -1.97); on average, girls are 
slightly less advanced in the pipeline. In this multivariate within-school HLM model, 
minority status is unrelated to progress in the mathematics pipeline {t = -1.25}. The chi- 
square statistics at the bottom of table 4 indicate that both average progress (the intercept, 
I>0) and its relationship with SES within schools (the slope, I>1) varies significantly 
between schools. This suggests that they are appropriate for modeling as outcomes in 
between-school HLMs. 

Model 2 . All measures of students’ demographic and academic backgroimd are 
significantly related to achievement in mathematics at the end of high school, as shown in 
Model 2A. Minority students and girls have lower achievement in mathematics; students 
from higher-SES families and those with higher math grades in ninth grade achieve at 
higher levels _{t = 6.19 and t = 24.22, respectively}. The magnitude of the SES and 
gender effects is small; the effects for minority status and math GPA are of moderate 
magnitude. However, once students’ coursework in mathematics is taken into account 
(Model 2B), the SES effect drops to non-significance {t = 1.48} and the influence of 
math GPA is more than halved (from .338 to .131, t = 1 1.07).'^ Minority and gender 
effects on achievement are relatively unchanged between Models 2A and 2B (i.e., the 
achievement gaps between male and female students, and between minority and majority 
students) are unaffected by their progress in the mathematics pipeline {t = -16.10 and t = 
-6.21 respectively}. Students’ progress in the mathematics coursework pipeline is 
strongly related to achievement in this multivariate model (I> = .658, t = 45.74). 

Regardless of social background and students’ academic status at the beginning of high 
school, and regardless of the high school they attend, students’ achievement in 
mathematics is strongly influenced by how far in the math pipeline they get.'^ In this 
subject, students achieve at higher levels when they take more advanced courses. Of 
course, students’ progress in the pipeline is itself a function of their social and academic 
backgrounds (as seen in Model T of table 4). As our model in figure 1 suggests, students’ 
demographic characteristics and their ability at the beginning of high school have both 
direct and indirect effects on achievement; the indirect effects “pass through” the courses 
students take during their high school career. A major objective here, however, is to 
investigate how the structure of the high school mathematics curriculum influences 
achievement. To pursue this question, we turn to our Level-2 HLM school effects 
models. 

Between-school HLM models . Results of our investigation of curriculum structure effects 
on students’ progress in the mathematics pipeline are displayed in table 5 (Model 3). As 
we explained earlier, this analysis includes the within-school model on this outcome 
shown in Model 1 of table 4. We investigate two outcomes here: (1) the adjusted school 
average of the highest math course students complete (I>0) and (2) the relationship 
between SES and coursework (I>1). We interpret these two outcomes as measures of 
effectiveness and equity. 



Two types of school effects are considered: (1) school composition and structure (which 
we use to control for academic and social differences among schools) and (2) curriculum 
structure. The emphasis here is on the latter. We began the modeling process with a full 
set of controls for school selectivity (average SES, school minority concentration, the 
proportion of low-achieving students in the school, school size, and school sector) on 
both the effectiveness and equity outcomes. However, in the models displayed in table 5 
we dropped several control variables due to non-significance. We followed a rule 
favoring model parsimony to maximize the stability of our HLM analyses (although in 
each model we retained a full and identical set of curriculum structure measures). 






Table Between-school HLM model of highest mathematics course completed 

(n = 3,430 students in 184 schools) 



Model 3 

Highest Mathematics Course Completed 


y Coefficient 


Random effects 






Intercept (Po) 


Intercept 




.041 


Composition and structure 


Average SES 




.163* 


Proportion of low-achieving students 




-.125* 


Catholic school 




.107 


NAIS school 




.275* 


Curriculum structure 


Courses offered below algebra 




-.041 


Mean, credits taken below algebra 




-.258* 


SD, credits taken below algebra 




.020 


Calculus offered 




.243* 


SES/coursetaking slope (Pi) 


Intercept 




.026 


Composition and structure 


Average SES 




.076* 


Curriculum structure 


Courses offered below algebra 




-.002 


Mean, credits taken below algebra 




-.017 


SD, credits taken below algebra 




. 102 * 


Calculus offered 




-.016 


Fixed Effects 


Gender (female) (P 2 ) 




-.056* 


Minority status (P 3 ) 




-.034 


Mathematics GPA, grade 9 (P 4 ) 




.310* 


The Chi-Square Table 




Estimated Parameter 


Degrees of 




Parameter Variance (Tau) 


Freedom 


Chi-Square 


Intercept .075 


175 


640* 


SES/Coursetaking Slope .02 1 


178 


209 



♦p<.05 



Note: Minority concentration and size were dropped from the intercept model. Minority concentration, size, 
sector, and proportion of low-achieving students were dropped from the SES/Coursetaking slope model. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, High School 
Effectiveness Study (HSES:90/92). 
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Average pipeline progress . Students move farther in the mathematics pipeline if they 
attend schools with higher average SES and fewer low-performing students {t = 3.56 and 
t = -3.87 respectively}. Although there were large observed sector differences shown in 
table 2, in these multivariate and multilevel models differences in pipeline progress 
between public and Catholic schools are small and non-significant {t = 1.19}. However, 
students in NAIS schools move significantly farther in the mathematics course pipeline 
than their public school coxmterparts (gamma = .275, t = 2.23). Sector differences in 
pipeline progress all but disappear once differences in factors such as curriculum 
structure are taken into accoxmt. The effects of curriculum structure are evident in this 
model. On average, the level of students’ progress in the mathematics course pipeline is 
lower when the school’s students take more low-level courses (those below algebra, t = 
5.27). On average, students move farther in the mathematics course pipeline when 
calculus is available in the school. 

There are few school effects on the SES/coursetaking slope. Coursetaking is less 
equitably distributed in higher-SES schools (gamma = .076, t = 2.20). Schools are also 
less equitable if the variability in low-level courses taken is higher (recall that positive 
effects increase the SES slope, which measures increased inequity, t = 2.45). However, in 
schools that offer no low-level courses (and hence students complete no low-level 
coursework), there is no significant social inequity (gamma = -.026, t = 0.28). Although 
measures of curriculum structure influence the intercept (itself a mean), a distributional 
measure (the SD of low-level credits) had an effect on the social distriliution outcome. 
Fixed effects in this model (gender, minority status, and mathematics ability) are very 
similar to those shown in table 4. 

How well has this school-effects model explained variance in average pipeline progress 
and the SES slope? We estimated this by comparing the taus fi-om the within-school 
models (in table 4) to the taus shown in the bottom panel of table 5. For the school 
average of the highest mathematics course completed, 77.4 percent of the between-school 
parameter variance has been explained [(.332 - .075)/ .332]. For the social distribution of 
pipeline progress, however, the model explained 22.2 percent of the variance. The 
difference in proportions of variMce explained for the two outcomes is attributable, at 
least in part, to the fact that slopes are generally estimated with considerably lower 
reliability (see technical note 14). 

Average mathematics achievement . The between-school HLM models with mathematics 
achievement as an outcome are shown in table 6. The within- school model is identical to 
that shown in Model 2B of table 4, where students’ progress in the mathematics course 
pipeline is included as a control. Along with gender, minority status, and nin th grade 
math GPA, this variable is estimated as a fixed effect. Again, the four fixed effects are 
virtually xmchanged between Level- 1 (table 4) and Level-2 models (table 6).‘^ The 
distinction between Models 4A and 4B relates to the variability in coursetaking. Model 
4A includes the four curriculum structure variables we included in Model 3 of table 5, 
whereas Model 4B adds the school average of the highest mathematics course completed 
and the standard deviation of that measure to the existing set of curriculum structure 



Table 6.— Between-school HLM models of mathematics achievement in grade 12 

(n = 3,430 students in 184 schools) 





Model 4A 


Model 4B 




Mathematics Achievement 


Mathematics Achievement 




Without Aggregate 


With Aggregate 




School Coursetaking 


School Coursetaking 




y Coefficient 


Y Coefficient 


Random Effects 






Intercept (Pq) 






Intercept 


-.001 


-.001 


Composition and Structure 






Average SES 


.358* 


.281* 


Proportion of Low-Achieving Students 


-.125* 


-.088* 


Curriculum Structure 






Courses Offered Below Algebra 


-.027 


-.017 


Mean, Credits Taken Below Algebra 


-.101* 


.002 


SD, Credits Taken Below Algebra 


-.003 


-.024 


Calculus Offered 


.114 


-.023 


Mean, Highest Mathematics 




.240* 


Course Completed 
SD, Highest Mathematics 




.081* 


Course Completed 






SES/Mathematics Achievement Slope (Pi) 






Intercept 


-.049 


-.152 


Composition and Structure 






Average SES 


.023 


-.004 


Curriculum Structure 






Courses Offered Below Algebra 


-.016 


-.005 


Mean, Credits Taken Below Algebra 


-.031 


-.003 


SD, Credits Taken Below Algebra 


.086* 


.110* 


Calculus Offered 


-.017 


-.020 


Mean, Highest Mathematics 




.073 


Course Completed 
SD, Highest Mathematics 




-.026 


Course Completed 






Fixed Effects 
Gender (Female) (P2) 


-.141* 


-.141* 


Minority Status (P3) 


-.303* 


-.302* 


Mathematics GPA, Grade 9 (P4) 


.131* 


.131* 


Highest Mathematics Course Taken (P5) 


.656* 


.658* 



SOURCE: U.S. Department of Education, National Center for Education Statistics, High School 
Effectiveness Study (HSES: 90/92). 
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Table 6.— Between-school HLM models of mathematics achievement in grade 12— 
Continued 



Chi-Square Table 



Parameter 


Estimated parameter 
variance (Tau) 


Degrees of freedom 


Chi-square 


Without Coursetaking, Model 4A 


Intercept 


.077 


177 


1048* 


SES/Achievement Slope 


.031 


178 


208 


With Coursetaking, Model 4B 


Intercept 


.067 


175 


924* 


SES/Achievement Slope 


.028 


176 


217* 



*p<-05 



Note: Minority concentration, size, and sector were dropped from the intercept model. Minority 

concentration, size, sector, and proportion of low-achieving students were dropped from the 
SES/Achievement slope model. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, High School 
Effectiveness Study (HSES:90/92). 



In both Models 4A and 4B, mathematics achievement is higher in schools with higher 
average SES and with a lower proportion of low-achieving students. With curriculum 
structure taken into account, we again dropped non-significant measures of school 
composition and structure (minority concentration, school sector, school size). 

Comparing effects of curriculum structure on average mathematics achievement in 
Models 4A and 4B, we see that average pipeline progress, and its variability, are strongly 
and significantly related to achievement. When average pipeline progress is not taken 
into account (Model 4A), achievement is lower in schools where the mean number of 
credits of low-level courses is higher. Though pipeline progress is related to calculus 
being offered (Model 3, table 5), here whether or not calculus is offered is unrelated to 
achievement. 

Thus, schools’ average achievement levels are higher when, on average, their students 
move farther in the mathematics course pipeline, but also when there is more variability 
in this pipeline measure within the school. There are few curriculum structure effects on 
the social distribution of mathematics achievement by SES. In Models 4A and 4B, 
variability in low-level coursetaking (i.e., average credits taken below algebra) is 
positively related to the SES/achievement slope. More variability in low-level 
coursetaking fosters a more inequitable social distribution of achievement in high 
schools. Aldiough they are strongly related to average achievement (effectiveness), the 
two measures of pipeline progress are unrelated to the distribution of achievement within 
schools (equity). 

The two between-school models shown in table 6 have explained over three fourths of the 
between-school variance in mathematics achievement: 76.6 percent for Model 4A; 79.4 
percent for Model 4B. However, the proportion of total variance in the SES/achievement 
slopes is only slightly affected by these between-school models (i.e., only 9.7 percent has 
been explained in Model 4B). 



Discussion 



Summary of Findings 

Observed differences . The number and type of mathematics courses offered and taken in 
high school are associated with many characteristics of students and schools. In our 
descriptive analyses, several background characteristics were associated with less 
progress in the curriculum (being black or Hispanic, being from a lower-SES family, 
earning lower math grades early in high school, being female). A similar pattern was 
found for high school curriculum structure. Schools with more minority students, with 
lower average SES, and with more low-achieving students demonstrate less average 
progress through the mathematics course pipeline. Average progress also differed by the 
school structural characteristics of sector (private schools show more progress) and size 
(students in smaller schools make more progress). However, these analyses did not take 
account of interrelationships between various background characteristics. 



All of the students in this sample stayed in high school until graduation. Comparisons of 
HSES students who were included and excluded in this study (appendix A) show the 
excluded sample to be disadvantaged in several respects (e.g., lower SES, lower grades in 
ninth grade — see appendix table A-1). Those comparisons provide evidence to suggest 
that the association between background and coursetaking in this study would be even 
larger if our sample included students who dropped out of high school. Thus, our 
estimates of such social stratification in these outcomes probably represent a lower bound 
for the full high school population. 

Determinants of students’ progress through the high school mathematics curriculuTn . 
Individual students’ achievement in mathematics is associated strongly with then- 
progress through the mathematics curriculum. Even in multivariate models which take 
account of students’ social backgroimd and academic status at the beginning of hi gh 
school, students’ progress in the mathematics course pipeline is the strongest single 
predictor of their achievement — ^twice the strength of any other factor. 

The association between coursetaking and achievement demonstrated in other research 
motivated our attempts to identify its determinants. In particular, we wanted to know 
whether the number and type of courses the school offers influences how far throu gh the 
standard academic mathematics sequence students get, and ultimately how both these 
factors influence achievement. Thus, the focus of this study was on the structure of the 
high school math curriculum and how it influences both students’ progress in the 
mathematics pipeline and also their achievement. 

Two features of the high school mathematics curriculum were foimd to be especially 
influential in determining how far students get. One element targets the low end of the 
curriculum: more coursetaking activity at the low end of the curriculum (below the level 
of Algebra I) leads to students making less progress. On the other end, we found that 
students move farther through the curriculum in schools that offer calculus. Further, when 
there is less variation in the number and type of low-level courses taken, the courses 
students take are more equitably distributed by family social class. Our multivariate 
analyses indicated that in schools whose curricula include fewer comses taken below 
algebra and more high-end offerings (especially calculus), students progress farther in the 
mathematics curriculum.'® 

Curriculum structure and achievement . In general, we conclude that the findings of this 
study provide evidence that is consistent with the constrained curriculum approach laid 
out at the beginning of the report. When the high school mathematics curriculum is 
structured by limiting low-end course options, students typically move farther through the 
curriculum and average achievement in the school is higher. However, more variation 
within schools in how far the students get in the pipeline, on average, is associated with 
higher achievement, not lower. Although average progress in the mathematics pipeline 
was unrelated to social equity in the distribution of achievement, more variability among 
low-level credits in that curriculum resulted in more inequality in achievement within 
schools by student SES.'® 



Sector, curriculum, and achievement . In this study, there were few significant effects of 
school sector on the outcomes. Although no-difference findings typically are not 
discussed in a report such as this, we highlight our non-findings regarding school sector 
for several reasons. Several other studies about the high school curriculum have used a 
school sector focus (e.g., Bryk, et al 1993; Lee & Bryk 1989). We also used sector 
differences as a guiding framework to structure our review of the literature. In addition, 
in this sample we found observed sector differences in curriculum, reported in the 
descriptive information in table 2. 

We offer two possible explanations for the lack of sector differences here. The first 
focuses on our analytic models. Once curriculum structure was included in the 
multivariate models, in general the effects of school sector (i.e., comparisons of Catholic 
and NAIS schools to public schools) disappeared. Of course, academic organization (or 
curriculum structure) represents a major difference between private and public secondary 
schools, as we described at the beginning. Therefore, controlling for structural differences 
explained away sector differences. A second explanation focuses on our sample. Because 
all of the schools in the HSES sample are located in or near America’s largest cities, 
schools in rural areas, those in and around towns and villages, and schools in mid-sized 
cities are not represented. It is also possible that the difference between this sample of 
schools and students and samples used in prior studies of this topic (that were closer to 
being nationally representative) accounts for differences between our findings and those 
of other studies with a similar focus. 

Some Implications of These Findings 

Composition . What kinds of students go to which types of schools, and what happens to 
those students in their schools, are fundamental questions embedded in school effects 
studies such as this one. In this study, we found strong effects for school social and 
academic composition. The HLM models took into account many social and academic 
characteristics of students, as well as the structure of the high school curriculum. 
However, even when these controls were included, students’ progress in the mathematics 
course pipeline and their achievement were shown to be adversely affected when there 
are high proportions of low-achieving students in the school, even when students’ own 
academic status at the beginning of high school is taken into accoimt. 

This finding is consistent with conclusions made by Barr and Dreeben (1983) and by 
Rutter and his colleagues (1979). Although those two studies examined learning in 
schools with different contexts, the authors of both studies described the detrimental 
effects on overall achievement when there are large proportions of low-performing 
students in an educational setting. The context for Barr and Dreeben’s study was first- 
grade classrooms; Rutter et al. studied iimer-city London secondary schools. 

We found that social as well as academic composition is important; the social-class level 
of the school influences achievement, even when several academic characteristics of 
students and schools were taken into account. This finding touches on the issue of access. 
U.S. students are mapped to high schools either through residential location, by parental 



choice among public schools, or through parents’ and students’ interest, ability, and 
willingness to consider private schools. Residential location, by far the most common 
determinant of school social and academic composition, is associated with family 
income. Increasingly, students and their families may choose the high school they attend. 
Choice of school, as well as willingness to exercise choice, is also related to family 
background (Carnegie Foundation for the Advancement of Teaching 1992; Lee, 
Croninger, & Smith 1994). The issue of access involves equity between schools, whereas 
the focus on equity in this study was within schools. Nevertheless, our findings 
underscore the importance of school social composition in dete rminin g average 
coursetaking and average achievement, above and beyond the structure of the curriculum. 

Low-level mathematics courses . Most studies of curriculum structure and achievement 
have focused on the high end of the mathematics curriculum. Our investigation of schools 
that offer large numbers of low-level courses — and the association of this type of 
cumculum with students’ coursetaking and achievement — are new. We demonstrate that 
when schools offer many low-end courses, students don’t move very far into the higher 
end of the curriculum. When schools offer fewer of these courses, students take fewer of 
them (see techmcal note 16). Instead, they take more advanced courses, the type of 
courses that much research (including this study) has shown to be associated with 
achievement (Lee, et al [1993] review this research). Virtually every high school, district, 
and state requires at least two years of mathematics to graduate, and some require more. 
Thus, students have to take some mathematics courses. The question is, “Which ones do 
they take?” 

Although taking low-level mathematics courses is associated with lower achievement, the 
curriculum in comprehensive high schools typically contains many such courses. The’ 
rationale for offering these courses may be that high school staff feels a responsibility to 
respond to perceived or actual student demand. When such courses are available, 
however, many students choose (or are counseled into) them. Some students are highly 
motivated or follow advice from parents who are well aware of the long term 
implications of taking demanding courses. Other adolescents, however, may not be 
academically ambitious or prepared to dive into demanding academic courses, and some 
parents may not have the experience and knowledge to offer guidance in this respect. The 
results of this study are consistent with conclusions made by Philip Cusick, who 
discussed the implications of extensive student choice in high schools that offered a 
broad curriculum with many undemanding courses. He called this system: ...electiveness 
accompanied by an open, non-tracked system... There are high school students who are 
either very mature or who have sufficient parental guidance to help them make their 
choices. But for those who are neither mature nor receiving any parental guidance, such a 
system may further disadvantage the already disadvantaged (Cusick 1983, p.76). 

We also found that high school curriculum stmcture influences social equity in 
achievement. When there is greater variation in a school’s pattern of mathematics courses 
taken below Algebra I, there is more inequity in both coursetaking and achievement 
within the school. It is often socially disadvantaged students who take large numbers of 
low-level courses (Lee & Bryk 1988, 1989; Lee, Smith, & Croninger 1997; Oakes 1990). 



These results, coupled with findings that demonstrated that schools where students took 
many low-end courses had lower achievement, suggest a direction for reform of the 
mathematics curriculum in U.S. high schools. High schools might consider the impact of 
offering no more than a very small number of courses below the level of Algebra I. These 
are, in essence, remedial courses. Taking such courses should be an unusual event, 
intended only as remediation to prepare students to take academic math courses.*^ 

Perhaps schools and districts might want to re-evaluate whether students should be able 
to satisfy graduation requirements with low-level courses. If these types of courses meet 
these requirements, students can graduate without having taken any academic courses. 

Specialized vs. constrained curriculxim, revisited . The findings of this study are consistent 
with the constrained curriculum approach in high school mathematics. However, the 
specialization model remains dominant in U.S. comprehensive high schools. Historically, 
the comprehensive public high school has been an institution that under one roof serves 
students with a wide range of abilities, interests, and future plans (Cusick 1983; Powell, 
et al 1985). The need to serve a diverse group of students in U.S. schools is as important 
today as it was at the begiiming of this century. There are, however, several modes of 
meeting this goal. U.S. public high schools have traditionally approached this task by 
offering separate programs to diverse groups of students (Oakes 1985; Powell, et al 
1985). Catholic high schools, with a similar tradition of educating socially and academic 
diverse students and families, historically approached the same task rather differently 
(Bryk, et al 1993). 

This curriculum full of specialization and choice became the dominant mode of public 
secondary school design when young people could leave high school and move into 
relatively high paying jobs that required few skills learned in school. As Mumane and 
Levy (1996) point out, however, the workplace that young people now face looks quite 
different. These authors conclude that all students need solid mathematical skills to 
function in the workplace of the 21st century, whether or not they are planning to attend 
college, whether or not they are intellectually curious, and whether or not they are highly 
motivated to engage in academic work while they are in high school. Regardless of future 
plans or present inclinations, all students now need a solid base in mathematics. 

Our results show that high schools where students take more mathematics courses offer 
fewer low-end courses. When schools offer mostly demanding academic courses, 
students advance further through the academic math curriculum. This pattern of student 
coursetaking is associated with their attaining higher achievement by fiie time they finish 
high school. In general, the results described in this report and the conclusions and 
implications we draw from them are consistent with several other of our published 
articles that focus on the general topic of curriculum structure (Lee 1993; Lee & Bryk 
1988, 1989; Lee, Croninger, & Smith 1997; Lee, Smith, & Croninger 1997). 

Other Conclusions From This Study 



How this study differs from other school effects studies . Because school effects studies 
investigate how the characteristics of schools influence the students attending those 



schools, analysis in such studies should reflect the nature of the research questions— they 
should be multilevel. Because this study falls into the school effects category, we used 
such multilevel methods. Methodologically, the study described in this report is similar to 
several other recent studies on this topic that we described at the beginning. However, 
this research differs in other ways from those studies. The comments in this section may 
be most relevant for researchers considering the HSES data for their own studies. In 
particular, we discuss some differences between the HSES data compared to NELS:88. 

To whom may we generalize? One difference is the sample. Compared to school effects 
studies that used NELS:88 data, the HSES data offer several advantages. One is larger 
within-school sample sizes. Statistical theory tells us that larger within-group samples 
lead to more precise estimates of the random parameters investigated in multilevel 
analysis (here, we estimated coursetaking and achievement, as well as relationships 
between both of these outcomes and student SES within each school). Another advantage 
is thatNCES has supplied school-level design weights with the HSES data. High-school 
weights are not included in the NETS data files; researchers who wished to pursue this 
type of study had to either confine their analyses to the student level, run multilevel 
analyses unweighted, or make their own school weights. None of these options is 
satisfactory for secondary data analysts, particularly those who may not be fully familiar 
with the NELS school “sampling strategy.” 

On the other hand, researchers who use the HSES data should give considerable thought 
to how the sample of schools and students used in this study departs from national 
representativeness. The original sample of HSES schools comes only from the areas in 
and around America’s largest cities, so schools in rural areas are excluded. Researchers 
also need to consider the procedure used to augment the sample of tenth graders within 
the HSES schools. In each high school in the HSES sample, the entire group of NELS 
students who were selected originally (in 1988) and were tenth graders (in 1990) was 
retained in the HSES within-school sample. However, augmentation s am ples of students 
in each school were drawn randomly from among the re mainin g tenth graders in the 
school. Thus, the combined sample of students within schools actually is two separate 
samples: one is a de facto sample (the original NELS students) and one is close to a 
randomly drawn sample. As these are actually two student samples, researchers cannot be 
sure how representative the combined sample of students in each school is of the 
population of 1990 tenth graders in that school. Additionally, the large amount of missing 
data in the HSES leaves researchers unclear about the representativeness of the 
operational sample even compared to the original HSES sample described in the 
codebook (Scott, et al 1996). These questions about sampling raise issues about the 
population of students and schools to whom results should be generalized. 

Statistical control for ability . The structure of the HSES dataset, and how it differs from 
that of the “parent” NELS data files, may have influenced the findings from this study. 

The base year of HSES is 1990, when students were near the end of the tenth grade. Data 
were collected for the first time from the augmented portion of the student sample half 
way through high school, compared to the full NELS (for which data are available from 
students before they entered high school in 1988). Thus, for only part of the HSES 



sample are test scores available at high school entry. If researchers wish to measure 
achievement gain over the four years of high school, which is necessary if they wish to 
assess the full impact of attending high schools on students (the essence of a school 
effects study), then the HSES dataset as a whole will not allow this kind of investigation. 
Our approach to this problem was to include a proxy measure of mathematical ability, the 
grades students earned in math during their first year of high school. Of course, use of 
this proxy factors out a quarter of students’ experiences in high school. 

We suggested that this measure was a reasonable proxy for mathematical ability. On the 
other hand, our measure is clearly not as good as a score on a test of achievement that is 
on the same scale as the outcome measure. We recognize that this control for 
mathematical ability at the start of high school may offer imperfect adjustment for this 
important personal characteristic. In addition to tapping student ability, course grades 
usually reflect school grading policies, relative student performance, and non-academic 
behavior. The lack of objective ability measures on students just before they enter high 
school, particularly achievement, is a drawback of using HSES data for school effects 
study compared to NELS, where test scores are available on virtually all students at 
eighth grade. 

Our conclusions here are identical to those made by Lee and Bryk (1989), who 
commented on their research using the High School and Beyond data:^° “There is always 
the possibility, of course, that the estimated school effects may be somewhat different if 
we had a better measure of the initial ability of students” (1989, p.l86). Although we 
could have used the measure of mathematics achievement at the end of tenth grade as a 
statistical control for ability, we suggest that by doing this we would be controlling away 
even more of the school effects we wish to estimate. 

Were sophomore achievement used as a proxy for differences in ability 
on entry into high school, the effects of school organization on the 
social distribution of achievement would be almost entirely adjusted 
away because much of these distributional differences are likely to be 
in place by the end of the sophomore year (Lee &. Bryk 1989, p. 177). 

A new measure of curriculum structure . The variable we used to operationalize students’ 
progress in the mathematics pipeline represents an innovation in several respects over 
past efforts to measure the construct of coursetaking. First, rather than focusing only on 
the number of mathematics courses taken (or even the number of academic courses 
completed), our measure focuses on the most advanced course students have taken in 
high school. Thus, this measure serves as an indicator of both the quantity and the content 
of courses taken. Second, because we drew information about coursetaking from 
students’ transcripts rather than from self-reports, the measure is quite reliable. Third, the 
school-level aggregate of this measure (and its standard deviation) represents one of 
several indicators (or perhaps a byproduct) of the structure of the high school 
mathematics curriculum. Hopefully, our efforts here represent an advancement in 
conceptualizing high school curriculum structure. Other researchers should consider this 
new indicator and perhaps apply the idea to other subjects in the curriculum. 



The effects on learning are indirect . Both our findings and our conceptualization of how 
Ae smicture of the high school mathematics curriculum influences students’ achievement 
in this subject (from figure 1) suggest an alternative approach. Our findings indicate that 
curriculum structure influences achievement mostly indirectly, following a two-stage 
process. In the first step, what courses a school offers (how many of which type) 
determines in large part what courses students take, even after accounting for their 
demonstrated performance in the subject. In the second step, the number and type of 
courses students take determine their achievement. Research on this topic might focus 
more fimitfiilly on investigating indirect effects such as these, rather than the typical 
approach of estimating only direct influences on achievement. Actually, it is usually total 
effects that researchers what to know, represented by the combination of direct and 
indirect effects. 

Final Comments 



Our final comments are both substantive and methodological. Our results suggest that 
where students go to high school has implications for their success. The composition of 
some high schools, with large numbers of low-achieving students and large proportions 
of economically disadvantaged students, is associated with lower student achievement 
(above and beyond the disadvantage individual students experience from low SES and 
low achievement status at entry). The implications from the findings of this study also 
pertain to possible reform of the high school math curriculum, at both low and the high 
end. The results may also have implications for the curriculum in elementary and middle 
schools. Mathematics is a cumulative subject requiring mastery of lower-level concepts 
before more advanced topics can be undertaken. To reach a skill level in mathematics 
during high school that is recognized as useful after graduation, in general students 
should be computationally and conceptually able to take Algebra I at high school entry 
(i.e., in the 9th grade). 



In at least three respects, this study represents an innovation compared to other recent 
studies of curriculum structure (some of them our own). One contribution is the new 
measure of cumculum structure, one that adds a new dimension to this work. Another 
contribution is the larger samples of students within schools compared to other studies 
with NELS data. Larger within-group samples allow more precise estimates of the 
constructs that are the focus of any study of this type. A third contribution relates to the 
conceptualization of the process through which curriculum structure influences 
achievement, a process that is to a great extent indirect as well as direct. 

On the other hand, we see a few limitations to using the HSES data for investigations of 
this type. One is the smaller and potentially unrepresentative sample of high schools. 
Another is the lack of data on a substantial proportion of HSES students as they enter 
high school, as well as difficulties involved in estimating school effects only for the last 
half of high school. Clearly, researchers who are interested in pursuing investigations that 
focus on school effects need to consider these issues as they undertake their work and 
make decisions about the best data and methods to use. 



Technical Notes 



Although the words excellence and effectiveness have similar meanings in the 
context of this study, we use the first in reference to student outcomes and the 
second in reference to school outcomes. 

The initial baseline sample size was 9,146 students. After data collection, 
however, the effective sample size dropped to 7,642 (Scott, et al 1996). We began 
our sample selection with diis group. The excluded students had no questionnaire 
data. 

We created a new flag for students with complete mathematics achievement 
scores (both tenth and twelfth) because we found that the original test flag 
(S290TPFL) includes students Avith incomplete mathematics test information. 

In both the text of this report and in Appendix C (where we describe the 
construction of all variables used in the study), we provide somewhat more 
information than we would for a scholarly article about the same study (e.g., we 
have provided actual names of HSES variables for sampling and variables). 
Because these data are new and somewhat unfamiliar to researchers, even those 
who are familiar with NETS, we hope to make others’ analyses of HSES data 
somewhat easier than ours has been. 

Schools who belong to the National Association of Independent Schools are 
homogeneous in the sense that the schools typically are quite selective in two 
regards. First, the tuition at these schools are quite high (especially compared to 
Catholic schools), so the schools are selective in an economic sense. Second, 
many of these schools have academic entrance criteria, so that many NAJS 
schools are selective in an academic sense. The title “independent” is quite 
appropriate; the schools are almost entirely autonomous from external control. In 
other places, we have referred to NAJS schools as “elite private schools.” 

During the HLM estimation procedure, in the presence of unequal within-school 
sample sizes, each school’s contribution is weighted proportionally to its 
statistical “precision.” As the within-school sample size decreases, so does its 
precision. Consequently, such schools (with fewer than 5 sampled students) are 
downweighted during the estimation process. Eliminating such schools from the 
analysis, although not absolutely necessary, tends to improve the efficiency of the 
algorithm (Bryk & Raudenbush 1992). 

Group-mean centering is a scaling alternative to grand-mean centering. In the 
former, the option we employed in this study, the within-school predictors are 
centered around their corresponding school means. This results in intercepts that 
can be interpreted as the adjusted mathematics progress (or mathematics 
achievement) for “average” students in the schools (where “average” is relative to 
the schools’ students). In the latter, the 'within-school predictors are centered 



around the single grand mean. This results in intercepts that can be interpreted as 
the adjusted mathematics progress (or math achievement) for “average” students 
(where “average” is defined independent of schools). 

The piupose of using effect sizes (measured in SD units) is that they are 
consistent across outcomes measured in different metrics. Thus, they represent a 
standardized measure of the strength of a statistical relationship. Substantive 
interpretation of effect sizes suggests that effects over .5 SD are considered large, 
those between .3 and .5 SD are moderate, those between .1 and .3 SD are small, 
and those below .1 SD are trivial (Rosenthal & Rosnow 1984). 

Naturally, such claims about the students in a school (e.g., all students in the 
school complete calculus) depend upon the randomness of the within-school 
sample. Strictly speaking, all students sampled from the school completed 
calculus. The larger within-school sample sizes available in HSES minimize the 
likelihood that such patterns are subject to substantial sampling error. 

The average number of completed credits below algebra is a rather gross estimate 
of average coursetaking behavior in the school in large part because of the severe 
positive skew within the school: in many schools a large proportion of the 
students complete no low-level coursework in mathematics. Hence, the school 
average combines students with no low-level coursework and students with 
possibly many years of low-level coursework. 

There is a distinction between within-category variation (captured by the SDs of 
the means for each of the three groups: low, middle, and high) and within-school 
variation (captured by the within-school SDs). The first measure of variation is an 
estimate, for example, of the similarity of all high-progress schools (e.g., high- 
progress schools as a group offer on average fewer low-level mathematics courses 
[mean number of courses = 2.1, see table 2], but there is substantial deviation 
from that mean within the category of high-progress schools [standard deviation 
of the means = 1 .9, see table 2]). In other words, there are some high-progress 
schools which offer fewer low-level courses and some high-progress schools 
which offer more low-level courses. In addition, we consider the within-school 
SDs which are a measure of the variability around the school mean, or a measure 
of the similarity of students within the school. 

Taking social and academic background into account in these within-school 
HLMs has increased the lambda reliabilities of the intercepts somewhat: .806 for 
the highest course taken (Model 1), .775 for mathematics achievement in Model 
2A, and .852 for achievement in Model 2B, compared to .770 and .719 for the two 
measures as shown in table 3. The SES/outcome slopes are much less reliable: the 
reliability of the SES/coursetaking slope is .237, and the SES/achievement slope 
reliability is .260 for Model 2A and .329 for Model 2B. These reliabilities are 
comparable to those reported in other school-effects studies that consider slopes 
as outcomes. 



The within-school HLM models shown in table 4 are rather different from OLS 
(ordinary least squares) regressions, in that the hierarchical structure of these 
models takes systematic differences between high schools into account. This is a 
major advantage of HLM over OLS. 

The measure of low-level courses offered had almost no effect, whereas the 
measure of the number of credits in low-level courses completed was significantly 
related to the outcome. It is possible that the differences in effects from the two 
measures (which supposedly tap the same general construct) relate to the source 
of the information for each. Principals may not have an accurate picture of the 
courses actually offered in any given year (but rather may be reporting on those in 
the school’s course catalogue). On the other hand, students can only take courses 
that are actually taught. Thus, our credits measure represents a more accurate 
accounting of what a school’s mathematics curriculum actually represents. 

We also ran a set of HLM analyses with the same two sets of curriculum structure 
controls shown in table 6, but where the within-school model was identical to that 
shown in Model 2A of table 4. As the results were very similar to those shown in 
table 6, we have not included them here. 

Our multivariate HLM models include several measures of a school’s curriculum 
structure. As indicated in Appendix C, the measure of courses offered was drawn 
from principals’ reports, whereas the measures of courses taken were aggregated 
from student reports. These two measures, which tap the same construct, are 
highly correlated. It is likely that the collinearity between high schools’ offering 
more courses and a large amount of coursetaking activity in those courses is 
measuring the same thing. Because we use the aggregate measure of student 
coursetaking as a proxy measure of offerings (i.e., high school curriculum 
structure), we draw conclusions related to schools’ curricular offerings. Lee and 
Smith (1995), Lee, Smith, and Croninger (1997) and Lee, Croninger, and Smith 
(1997) follow this same procedure for measuring curriculum structure. 

More variation within schools suggests the possibility of more students at the low 
end and/or the high end of the mathematics course pipeline. In general, larger 
within-school variation is associated with lower average pipeline progress (r = - 
.36). However, after controlling for average progress, the fact that greater 
variation leads to higher achievement suggests that the greater variation in this 
sample is most likely associated with more students at the higher end of the 
pipeline. 

We have chosen to put a substantive meaning on this finding. However, it is 
possible that there is an artifactual explanation. The findings for social 
composition are similar to those reported in many studies (including our own), but 
our findings about the academic composition are unusual. It is possible that this 
context effect is due, at least in part, to our use of the aggregate of our proxy 



measure of student ability — ^ninth grade mathematics GPA — as a measure of 
academic composition. 

The typical practice for remediation in Catholic high schools is quite different 
from remedial courses in comprehensive public high schools (Bryk, et al 1993). 

If, for example, a student entered a Catholic high school without the skills to 
succeed in Algebra I, the student would typically be enrolled in that class as a 
ninth grader. However, he or she would also be enrolled in a remedial (and non- 
credit) class simultaneously; the student would be taking two math classes at the 
same time. Remediation would continue until the student was able to succeed in 
the regular academic mathematics course on his/her own. In these school settings, 
remedial courses do not serve as alternative ways to satisfying graduation 
requirements in any subject. Rather, their purpose is to bring students up to 
competence to succeed in academic courses. Obviously, this type of program 
assumes that, eventually, all students can succeed in an academic curriculum. 

High School and Beyond (HS&B) is a nationally representative longitudinal study 
of high school students’ educational progress conducted by the U.S. Department 
of Education. Data on base year samples of high school sophomores and seniors 
were collected in 1980, with the same students followed with biennial data 
collections. Researchers recognized a limitation in the HS&B design and data 
structure: there were no data available on students as they entered high school or 
during the early years of high school. To a great extent, this difficulty with HS&B 
gmded the design of NELS:88. The HSES data suffer, however, from some of the 
same difficulties in design as HS&B. 
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Appendix A: Analysis of differences between included and excluded members of the 
analytic sample 



Rationale 



The purpose of this analysis is to examine potential differences between the students and schools 
included in our analytic sample and those students and schools that were excluded from the 
analysis through our sample selection criteria. The discussion is intended to describe (1) the 
direction of mean differences between the included and excluded samples, (2) the effect of the 
missing data on our analyses, and (3) the generalizability of our findings. 

Analytic Sample 



Our investigation required information about student demographics, coursetaking behaviors, 
math achievement, and a number of school structure characteristics. We selected students with 
complete questionnaire information (10th and 12th), complete mathematics test information 
(10th and 12th), complete transcript information (9th, 10th, 11th, and 12th), and complete 
demographic information (race/ethnicity, socioeconomic status, and gender). We also selected 
students in schools with administrative questionnaires available (10th and 12th), and schools that 
were public. Catholic, or private schools affiliated with the National Association of Independent 
Schools (NAIS). Finally, we selected only those schools with 5 or more students who satisfied 
the above criteria. These selections resulted in an analytic sample of 3,430 students in 184 
schools. 

Excluded Sample 

The HSES Data File User ’s Manual (Scott, et al 1996) indicates that 9,146 students were 
selected for the 1990 baseline student sample (p. 33). This sample explicitly excluded students 
with the following characteristics: (1) part-time students; (2) students who transferred out of the 
school after the student roster was compiled; (3) students who were on the roster but not in tenth 
grade, or those who had been absent or truant for 20 consecutive days and were not expected to 
return to school; (4) physically or mentally disabled students who were imable to fill out the 
questionnaires or take tests; (5) students for whom English or Spanish was a second language 
and who lacked language proficiency to complete the questionnaires and tests; and finally (6) 
those students who were deceased (p. 34). Although 9,146 students were selected for the baseline 
sample, the User ’s Manual notes that five of these baseline students, three of whom filled out 
questionnaires, were foimd to have been sampled in error. These cases were included in the 
weighting, but were not included on the HSES data file (p. 33). 

Of the 9,141 students initially selected for the baseline sample, only 7,642 students completed 
questionnaires. These participating students constitute the effective student sample (see the HSES 
Data File User’s Manual, 1996, table 6. 1.1-1). Unfortunately, not all of these tenth-grade 
students remained in the same school between 1990 and 1992. Unlike the NELS:88 study design, 
HSES did not attempt to follow students who dropped out or transferred to a different school. In 
addition to these losses, not all of the tenth-grade students who remained in the same school 



between 1990 and 1992 chose to fill out HSES survey questionnaires in 1992. Those students 
who left school or chose not to participate in the 1992 followback were excluded from our 
analytic student sample. Similarly, 247 schools were selected for the baseline school sample, but 
only 242 participated in the baseline questionnaires. These schools constitute the effective school 
sample from which we drew our sample. 

The samples of 7,642 students in 242 schools provide the basis for our analysis of differences 
between those students and schools that were included in our analytic sample and those that were 
not. Since 3,430 students and 184 schools were included in our analytic sample, a maximum of 
4,212 students and 58 schools should be considered as the excluded group. Because the number 
of cases missing information was different for different variables, we include specific excluded 
sample values in appendix tables A-1 and A-2. 

Procedure 



We first identified all participating baseline students (7,642) using the SISTAT baseline status 
indicator. Students who were included in our analytic sample (3,430) were coded 1, and students 
excluded from the sample (4,212) were coded 0. We then used t-tests to examine potential 
differences between the included and excluded students for all of the continuous student 
variables used in our study: socioeconomic status, 9th-grade mathematics grade point average, 
highest mathematics course completed, and 12th grade mathematics achievement. We used chi- 
square tests to examine potential differences between student-level categorical variables: gender 
and minority status. Note that all tests of student variables were based on unweighted means. 

We followed a similar procedure to investigate potential differences between the 184 schools 
included in our analytic sample, and the remaining 58 schools that were not included in the 
analysis. We then compared the two groups of schools (included and excluded) on the school- 
level variables used in our study: average socioeconomic status, mean and SD of the 9th-grade 
mathematics grade point average, mean and SD of the highest level of mathematics courses 
completed, the average number of courses below algebra that a school offers, the mean and SD 
of the number of below algebra math credits taken by students, the mean and SD of 12th-grade 
mathematics achievement, school size, the number of students per school, the proportion of low- 
achieving students per school, school sector, a school’s minority status, and the percent of 
schools offering calculus. Note that these tests use school weights. 

Findings for Excluded Sample: Students 

The results of the comparisons clearly indicate that the included and excluded student samples 
are demographically and academically different (see appendix table A-1). The excluded sample 
of students includes a sigmficantly larger percentage of minorities [t=7.01 ] and is of 
significantly lower socioeconomic status [t=-8.65] compared to the students in the analytic 
sample. Excluded students also show significantly lower prior mathematics grades [t=-6. 71 ], and 
significantly lower 12th grade mathematics achievement levels [t=-2.03J. The highest level of 
mathematics completed by the excluded students is significantly lower than the highest level of 
mathematics completed by those students in the analytic sample [t=-19.49] . The findings in 



appendix table A-1 clearly indicate that those students who were excluded from the analytic 
sample are also more disadvantaged. 

Findings for Excluded Sample: Schools 

The differences between included and excluded schools appear less systematic than differences 
between included and excluded students (see appendix table A-2). In terms of school 
composition, the sample of excluded schools had a significantly larger percentage of minority 
students [t=3.13), but the included and excluded samples are not significantly different in 
average socioeconomic status [t= .67]. The proportion of low-achieving students in the excluded 
school sample was significantly lower than the proportion of low-achieving students in the 
analytic sample [t=-11.15]. This is a curious finding that is discussed below in more detail. 

In terms of structural characteristics, the average size of the excluded schools is significantly 
smaller than the average size of the schools in our analytic sample [t=-5.19]. The excluded 
schools also have significantly smaller within-school student samples [t=-7.93]. The analytic 
sample includes significantly larger proportions of Catholic [t=-2.35] Bnd NAIS private schools 
[t=-2.42]. Given that we explicitly excluded non-NAJS private schools from our analysis, it is 
also imsurprising that the two samples show a significant difference in terms of sector [t=11.90J. 

Appendix table A-2 indicates that included and excluded schools also show a number of key 
differences in the school composition and structure, and the structure of the mathematics 
curriculum. The most pronounced trend is that schools in our analytic sample tend to have 
significantly more variation in mathematics measures (prior grades [t=-8. 72], 12th grade 
achievement [t=-5.58], and highest course completed [t=-6.05]), although average levels of 
these outcomes do not vary significantly between groups [12^^ grade achievement, t=-1.01; 
highest math course completed, t=-.90]. A key exception to this trend is that the average of prior 
mathematics grades in excluded schools was significantly higher than the average prior grades in 
our analytic sample [t=3.68]. This is consistent with the earlier finding that excluded schools had 
lower proportions of low-achieving students. 

Although excluded schools have a significantly higher percentage of minority students [t=3.13], 
a characteristic that tends to be associated with higher levels of poverty, the excluded sample 
shows no SES deficit [t=.67]. Moreover, these excluded schools appear to be advantaged in 
terms of student grades (recall that the proportion of low-achieving students is the proportion of 
students with mathematics grades below C) [t=3.68]. Despite the indication of high student 
grades, excluded schools are much less likely to offer calculus compared to schools in the 
analytic sample which have higher proportions of students with low grades [t=-5.93]. These 
seeming inconsistencies may be due, in part, to the collection of schools excluded from the 
analysis. The unweighted excluded sample includes 37 public schools, 2 Catholic schools, 4 
NAIS schools, and 15 non-NAIS private schools. Further analysis revealed that the excluded 
public schools have lower SES [t=-2.48], larger minority enrollments [t=3. 76], lower average 
9th grade mathematics grades [t=-2.10], and lower average 12th grade achievement [t=-2.25] 
than their public school counterparts in the analytic sample. The presence of these two 
subsamples within the excluded group helps to. explain the high percentage of minority students 



and the lack of significant social class differences between included and excluded schools. 
Moreover, it seems that evidence of higher grades in the excluded schools could stem from 
several sources. The increased selectivity of non-public schools is likely to reduce the proportion 
of students with low mathematics grades and to increase the number of students with higher 
average early mathematics grades. Since non-public schools compose 69 percent of the weighted 
excluded sample, the influence of selectivity on mathematics grades is more pronounced in the 
set of excluded schools. 

Summary 

The purpose of this analysis is to examine whether systematic differences exist between those 
students and schools included in our ^alytic sample and those that are not. Ideally, the included 
and excluded samples would not differ. Such a finding would support the claim that our analytic 
sample was representative of the full HSES sample, and therefore generalizable to urban and 
suburban students and schools in the 30 largest U.S. metropolitan areas. However, we cannot 
make this claim. The analytic sample used for this study systematically excludes certain students. 

Although significant compositional, structural, and curricular differences exist between schools 
that were included and excluded from the analysis, these differences are not systematic, and do 
not suggest that excluded schools are either more advantaged or more disadvantaged. However, 
further analysis of the excluded sample suggests that excluded public schools were significantly 
more disadvantaged in terms of composition and curricular structure compared to public schools 
included in the analysis. This suggests that our multivariate analyses may not provide adequate 
control for select school characteristics. 

The HSES sample was designed to represent students and schools in the urban and suburban 
portions of the 30 largest U.S. metropolitan areas. The previous analysis suggests that our sample 
is generalizable to a slightly more affluent segment of students and schools within these areas. 

We recognize that sample selection always represents a balance between samples adequate for 
the type of analysis employed and generalizability. Because the full HSES sample itself is not 
representative of the U.S. high school population of students and schools, we conclude that our 
sample selection criteria are appropriate. 



Appendix Table A-1.— Means and sample sizes for student-level variables 



(Included, n = 3,430; excluded, n = 4,212) 



Variable 


Included students 
Mean 


Excluded students 
Mean 


Excluded students 

Variable-Specific 
Sample Size' 


Percent Female 


48 


49 


4,212 


Percent Minority 


30 


38* 


4,073 


Social Class 


.30 


.14* 


3,940 


Mathematics GPA, Grade 9 


2.46 


2.28* 


2,472 


Highest Math Course Completed 


5.69 


4.80* 


2,701 


12th-Grade Mathematics Achievement 


51.50 


50.33* 


932 



* p<.05 



I. Excluded sample means were calculated using excluded cases with available information. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, High School Effectiveness Study 
(HSES:90/92). 
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60 



Appendix Table A-2.— Means and sample sizes for school-level variables 



(Included, n = 184; excluded, n = 58) 



Variable 


Included schools 
Mean 


Excluded schools 
Mean 


Excluded schools 

Variable-Specific 
Sample Size' 


A. School Composition 


26 


46* 


56 


Percent High Minority Schools 


31 


14* 


58 


Percent of Low-Achieving Students 


.29 


.33 


57 


Average SES 


2.46 


2.28* 


2,472 


B, School Structure 


Percent Catholic Schools 


20 


9* 


58 


Percent NAIS Schools 


8 


2* 


58 


Percent “Other” Private Schools 


0 


58* 


58 


School Size 


1176 


616* 


58 


Student Count Per School ^ 


31 


21* 


58 


C. Structure of Mathematics Curriculum 


Percent Offering Calculus 


82 


44* 


42 


Mean, Courses Offered Below Algebra 


3.77 


3.70 


58 


Mean, Credits Taken Below Algebra 


.63 


.72 


42 


SD, Credits Taken Below Algebra 


.71 


.66 


42 


D. Student Achievement 


Mean, Mathematics GPA, Grade 9 


2.38 


2.65* 


42 


SD, Mathematics GPA, Grade 9 


.96 


.69* 


33 


Mean, Mathematics 12 Achievement 


50.77 


49.24 


49 


SD, Mathematics 12 Achievement 


12.21 


9.91* 


40 


Mean, Highest Mathematics Course Completed 


5.62 


5.47 


38 


SD, Highest Mathematics Course Completed 
* d<-05 


1.26 


.76* 


31 



1 . Excluded sample means were calculated using excluded cases with available information. 

2. This variable provides the mean number of students in each school in our sample. To obtain this number, we 
aggregated the student ID variable to the school level and created a new variable, which we called “count.” The 
mean of this variable for each group is presented here. 

SOURCE: U.S. Department of Education, National Center for Education Statistics, High School Effectiveness Studv 
(HSES:90/92). a yy 



Appendix B: A brief discussion of school- and student-level weights in the HSES data set 



The Need for Weights 

Because of the complex sampling strategies involved in the HSES data set, the development of 
school-weights was an especially complicated process: 

The difficulty in developing weights stemmed from the method of “selecting” 
the lOth-grade schools constituting the baseline sample. Tenth-grade schools 
were selected as a direct consequence of the fact that one or more NELS:88 
first follow-up sample members attended the school in 1990. HSES schools 
were not selected with known probabilities from an initial completed sampling 
frame. Therefore, to create school weights, it was necessary to determine for 
each HSES school the probabilities associated with students’ transition from a 
NELS:88 base year (1988) eighth-grade school to a particular lOth-grade 
school in 1990 (HSES Data File User’s Manual, 1996, p. 35) 

Selecting a School-Level Weight 

Three distinct baseline weighting procedures (employing distinct probability models) resulted in 
a set of school-level weights (see the HSES Data File User ’s Manual, 1996, for a complete 
discussion of the weights). These three weights are not highly correlated with one another. 
Weight 1 (the Spencer-Foran weight) and weight 3 (the Qian-Frankel weight) are most highly 
correlated (r = .5656), whereas weights 2 (the Kaufman weight) and 3 are only modestly 
correlated (r = .2855). Weights 1 and 2 have a rather low relationship to one another (r = .1496). 

In keeping with the low correlations between them, the weights perform in a differential fashion 
and “none of the weighting models provides consistently close agreement between population 
and sample estimates” (Appendix M, HSES Data File User ’s Manual, 1996. p. M-6). Our own 
preliminary exploration of the three weights in the context of simple HLM models of 12th-grade 
mathematics achievement was similarly nonconclusive. Although the size of coefficients 
changed imder the four conditions (imweighted and weighted separately by each of the three 
school-level weights), in general the significance levels did not. The chi-square statistics testing 
between-school variation in outcomes increased when weighted, but the significance levels 
remained similar. 

Neither our analyses nor those described in Appendix M of the HSES Data File User ’s Manual 
provided imequivocal evidence of the greater efficacy of a particular school-level weight. 
Consequently, the choice of a school-level weight seems to remain rather arbitrary. We decided 
to use the Qian-Frankel weight, in part because it is most strongly related to the other two 
(especially to the Spencer-Foran weight). 

Concerns \vith the Student-Level Weights 

Despite the indicated optimism — ^“Once school weights were developed, the calculation of 
student weights was a relatively straightforward process” (HSES Data File User ’s Manual, 1996, 



p. 36 >— the construction of the HSES student-level weights proved to be less than “straight- 
forward”: 

Given the way in which the HSES student sample was selected, students within 
a selected HSES school can be treated as two strata: students who were 
members of the NELS:88 sample and those who were not. Since all of the 
students who were members of the NELS:88 sample and who were enrolled in 
a HSES school were included in the HSES student sample with certainty, the 
probability P3 (the conditional probability that the lOth-grade student is in the 
HSES student sample, given that the student’s lOth-grade school is in the 
HSES school sample) for these students equals 1 . The students who were not 
members of the NELS :88 student sample were selected by systematic random 
sampling, which approximates simple random sampling without replacement. 

Therefore, P3, for these students is equal to their within-school selection 
probability {HSES Data File User’s Manual, 1996 , p. 40 ). 

While the conceptualization of the student sample as composed of two strata is straightforward 
enough, the assignment of P3 = 1 for those HSES students who were members of the NELS:88 
sample results in practical concerns for both (a) the overall student weights (used for descriptive 
purposes, factor analyses, preliminary OLS regression models, etc.) and (b) the within-school 
weight (used for HLM modeling). All but one of the student-level HSES weights (the exception 
being the student-level, 8-12 panel weight, S 288 PNWT) systematically downsize the original 
NELS:88 students within the HSES sample. This downsizing is not modest in magnitude, but 
essentially eliminates those cases from statistical analyses. Hence, the twofold strata are replaced 
with what amounts to a single strahim. 

Roughly one third (unweighted) of the entire HSES sample is drawn from the original NELS:88 
students. This is also true of our reduced study sample of 3430 . These relative proportions are 
true, however, for the unweighted samples only. Employing the HSES weights reduces the 
stratum of NELS:88 students to less than 5 percent of the sample. This alteration in the relative 
proportions of sample membership affects the magnitude and significance of parameter estimates 
in multivariate models: weighted and unweighted results are often quite dissimilar, much more 
so than with other NELS studies (e.g.. High School and Beyond, NELS:88). 

In their current state, the HSES student-level weights are not effective or useful. New weights are 
being constmcted that preserve both strata of the student sample. All analyses we present in this 
report involving student-level data are unweighted at the student level. This includes: (a) 
descriptive information on the student sample, and (b) HLM models (weighted using the school- 
level weight only — ^without the within-school student weight). 



Appendix C: Details of construction of variables and weights used in the study of 
curriculum structure and mathematics achievement 

All variables used in this study were drawn from the following HSES files: student, school, and 
transcript. 

Variables Describing Students: Outcome Measures 

12th Grade Mathematics Achievement. A continuous variable measuring IRT-estimated 
number right on the 12th grade mathematics test (S22XMIRR from student file) [M = 
51.50, SD = 15.22]. For multivariate analysis, normalized on students to a z-score metric 
[M = 0, SD= 1]. 



Highest Level of Mathematics Course Completed. Eight-level variable constructed from 
NELS/HSES student transcript data (student course credits in mathematics). The scale 
was created by selecting the highest level of mathematics course each student completed 
with a passing grade during his/her four years in high school. More detail on the logic of 
construction, and how the variable behaves in multivariate analyses, is available from the 
authors. The variable will be available on new releases of NELS and HSES data from 
NCES [M = 5.69, SD = 1 .66]. For multivariate analysis, normalized on students to a z- 
score metric [M = 0, SD = 1]. 

Level 1 : No Mathematics. Student took no mathematics courses in high school. 

Level 2: Non- Academic. General mathematics 1 or 2, basic mathematics 1, 2, or 3, 
consumer mathematics, vocational mathematics, or review. 

Level 3: Low Academic. Pre-algebra, algebra I stretched over two years, informal 

geometry. 

Level 4: Middle Academic I. Algebra I, plane geometry, solid geometry, unified 

mathematics 1 or 2, pure mathematics. 

Level 5: Middle Academic II. Algebra II, unified mathematics 3. 

Level 6: Advanced I. Algebra 3, algebra/trig, analytic geometry, trigonometry, 

probability, statistics, independent study. 

Level 7: Advanced II. Pre-calculus, introductory analysis. 

Level 8: Advanced III. Calculus, AP calculus, calculus/analytic geometry. 

Variables Describing Students: Backgroimd Characteristics 

Social Class (SES). Socioeconomic status (composite includes parents’ education, father’s 
occupation, family income, and several educationally-oriented household possessions 
[S2SES from the student file]), normalized on students to a z-score metric (M = 0, SD = 
1 ). 

Gender. Gender of student; coded 1 = female, 0 = male (recoded SI SEX from the student file). 
[48.2 percent are female, 51.8 percent are male] 

Minority Status. Race/ethnicity of student; coded African American, Hispanic, or Native 

American = 1 ; White or Asian = 0 (recoded SI RACE from the student file). [30.3 percent 
are minority, 69.7 percent are non-minority] 



9th Grade Mathematics Grades. A continuous variable indicating students’ grades (at the end 
of 9th grade) in the subject of mathematics (taken from the transcript file) [M = 2.46, SD 
= 1.01]. For multivariate analysis, normalized on students to a z-score metric IM = 0, SD 
= 1 ]. 

Variables Describing Schools: Mathematics Curriculum Structure 

School Mean, Highest Mathematics Course Completed. A school aggregate from student 
transcript data; the within-school average of the student-level measure: highest 
mathematics coxirse completed. This measure represents the average highest mathematics 
course completed for each school [average of the school means = 5.62, standard deviation 
of the school means = 1.09]. For multivariate analysis, normalized on schools to a z-score 
metric [M = 0, SD = 1]. 

School SD, Highest Mathematics Course Completed. A school aggregate from student 

transcript data; the within-school standard deviation of the highest mathematics coxirse 
completed. This measure represents the variability of students’ mathematics coursetaking 
in each school [average of the school standard deviations = 1.26, standard deviation of the 
school SDs = .46]. For multivariate analysis, normalized on schools to a z-score metric 
[M = 0, SD= 1]. 



Number of Mathematics Courses Offered Below Algebra. School-reported curriculum 
offerings, sum of six dummy-coded course offerings indicators: S1C75R2 (General 
Mathematics, grade 9), S1C75S2 (General Mathematics, grades 10-12), S1C75T2 
(Business Mathematics), S1C75U2 (Consumer Mathematics), S1C75V2 (Remedial 
Mathematics), S1C75W2 (Pre-Algebra), [M = 3.77, SD = 1.79]. For multivariate 
analysis, normalized on schools to standard deviation units [M = 2.20, SD = 1.00]. 

School Mean, Number of Mathematics Credits Completed Below Algebra. A school 

aggregate measure from student transcript data; the within-school average number of 
credits completed in the following mathematics courses [CSSC codes]: 270100 
(MATHEMATICS, OTHER GENERAL), 270101 (MATHEMATICS 7), 270103 
(MATHEMATICS 8), 270104 (MATHEMATICS 8, ACCEL), 270106 
(MATHEMATICS 1, GENERAL), 270107 (MATHEMATICS 2, GENERAL), 270108 
(SCIENCE MATHEMATICS), 270109 (MATHEMATICS IN THE ARTS), 270110 
(MATHEMATICS, VOCATIONAL), 270111 (TECHNICAL MATHEMATICS), 

2701 12 (MATHEMATICS REVIEW), 2701 13 (MATHEMATICS TUTORING), 2701 14 
(CONSUMER MATHEMATICS), 270300 (APPLIED MATHEMATICS, OTHER), 
270601 (BASIC MATHEMATICS 1), 270602 (BASIC MATHEMATICS 2), 270603 
(BASIC MATHEMATICS 3), 270604 (BASIC MATHEMATICS 4), 270401 (PRE- 
ALGEBRA), 270402 (ALGEBRA 1, PART 1), 270403 (ALGEBRA 1, PART 2), 270409 
(GEOMETRY, INFORMAL), [average of the school means = .63, standard deviation of 
the school means = .49]. This measure represents the average number of low-level 
mathematics courses completed for each school. For multivariate analysis, normalized on 
schools to standard deviation units [M = 1.26, SD = 1.00]. 



School SD, Number of Mathematics Credits Completed Below Algebra. School aggregate 

measures from student transcript data; the within-school standard deviation in the number 
of credits in the following mathematics courses [CSSC codes]: 270100 
(MATHEMATICS, OTHER GENERAL), 270101 (MATHEMATICS 7), 270103 
(MATHEMATICS 8), 270104 (MATHEMATICS 8, ACCEL), 270106 
(MATHEMATICS 1, GENERAL), 270107 (MATHEMATICS 2, GENERAL), 270108 
(SCIENCE MATHEMATICS), 270109 (MATHEMATICS IN THE ARTS), 270110 
(MATHEMATICS, VOCATIONAL), 2701 1 1 (TECHNICAL MATHEMATICS), 

2701 12 (MATHEMATICS REVIEW), 270113 (MATHEMATICS TUTORING), 270114 
(CONSUMER MATHEMATICS), 270300 (APPLIED MATHEMATICS, OTHER), 
270601 (BASIC MATHEMATICS 1), 270602 (BASIC MATHEMATICS 2), 270603 
(BASIC MATHEMATICS 3), 270604 (BASIC MATHEMATICS 4), 270401 (PRE- 
ALGEBRA), 270402 (ALGEBRA 1, PART 1), 270403 (ALGEBRA 1, PART 2), 270409 
(GEOMETRY, INFORMAL), [average of the school standard deviations = .71, standard 
deviation of the school SDs = .37]. This measure represents the variability of low-level 
mathematics coursetaking in each school. For multivariate analyses, normalized on 
schools to standard deviation imits [M = 1.88, SD = 1.00]. 

Calculus Offered. In order to explore the availability and influence of calculus in the 

mathematics curriculum, we considered data from three sources: (1) the HSES Course 
Offerings File, (2) the HSES School File, and (3) the HSES Student Transcript File. 
Unfortunately, the Course Offerings File proved to be a problematic data source with 
substantial missing data. Over 30 of the schools in our analytic sample were missing 
course offerings data, and data from other schools seemed highly unlikely (e.g., one 
school claimed calculus as the only math course available, another school was coded as 
offering three years of low-level, general math and one year of calculus). 

Data from the second and t hi rd sources resulted in two possible measures. The first 
variable used school responses (S1C75EE1 and S1C75FF1) to identify those schools that 
claimed to offer calculus. Unfortunately, 40 schools in our analytic sample were missing 
information about their calculus course offerings. Given the substantial amount of 
missing information, we decided to explore the possible use of aggregated student-level 
data. 

We constructed the second variable by first identifying all students whose transcript 
information indicated that they had taken calculus (students who scored at the highest 
level on the math coursetaking variable, MTHQUAL8). We then aggregated these scores 
to the school-level in such a way that if at least one student in the school indicated that 
they had taken calculus, the school was identified as offering calculus. One of the 
advantages of this method is that it identifies whether calculus was actually taken in a 
school, and not simply whether a calculus course was “on the books.” None of the 
schools in our analytic sample had missing data for this variable. 

Comparisons between these two measures of the availability of calculus revealed some 
inconsistencies: (1) there were schools claiming to offer calculus but no one in our school 



sample had elected it, and (2) there were schools claiming not to offer calculus but 
sample members had nonetheless completed a calculus course. In the first instance, the 
discrepancy could simply be due to sampling, but it could also reflect a course that is 
offered only “on paper.” In the second instance, the discrepancy might occur if a student 
went elsewhere (i.e., a nearby college) for calculus instruction. Additional inquiries led us 
to believe that the measure based on actual course election was better suited for use in the 
multivariate models. Consequently, the availability of calculus is coded 1 = at least one 
student in the within-school sample completed calculus, and 0 = no one in the within- 
school sample completed calculus [82 percent of schools offer calculus, 18 percent do 
not]. 

Variables Describing Schools: Composition and Structure 

School Sector. Two dummy variables (created from G10CTRL2, from school file): 

Catholic, coded Catholic = 1, public and elite private schools coded 0. 

NAIS, coded NAIS = 1, public and Catholic schools coded 0. 

In multivariate analyses, public schools serve as the comparison group [72.1 percent are 
public, 19.9 percent are Catholic, 8.0 percent are private]. 

School Mean, SES. A school-level aggregate; the within-school average of the student SES 
variable described above [average of the school means = .29, standard deviation of the 
school means = .49]. This measure represents the average SES of each school. For 
multivariate analysis, normalized on schools to a z-score metric [M = 0, SD = 1]. 

School Size. Continuous variable indicating the number of students enrolled in each school 

(S1C2 from the school file) [M = 1176, SD = 748]. For multivariate analysis, normalized 
on schools to a z-score metric [M = 0, SD = 1]. 

Minority Concentration. A dummy-coded variable (a sum of the percent Native American 10th 
graders [S1C27A], percent Hispanic 10th graders [S1C27D], and percent Black (not 
Hispamc) 10th graders [S1C27E], from the school file); schools with proportions of 
minority students 40 percent or above coded 1, others coded 0. [26 percent of schools are 
high minority, 74 percent are low minority] 

Proportion of Low-Achieving Students. This measure indicates the proportion of students in 
each school who earned below a ‘C’ in 9th grade mathematics [M = .31, SD = .18]. For 
multivariate analysis, normalized on schools to a z-score metric [M = 0, SD = 1]. 

School Mean, 12th Grade Mathematics Achievement. A school aggregate; the within-school 
average of 12th grade mathematics achievement [average of the school means = 50.77, 
standard deviation of the school means = 8.09]. This measure represents the average 12th 
grade mathematics achievement for each school. Used for descriptive purposes only. 

School SD, 12th Grade Mathematics Achievement. A school aggregate; the within-school 
standard deviation of 12th grade mathematics achievement [average of the school 
standard deviations = 12.21 , standard deviation of the school SDs = 2.47]. This measure 



represents the variability in 12th grade mathematics achievement for each school. Used 
for descriptive purposes only. 

School Mean, 9th Grade Mathematics Grades. A school aggregate; the within-school average 
of 9th grade mathematics grades for each school [average of the school means = 2.38, 
standard deviation of the school means = .41]. This measure represents the average 12th 
grade mathematics grades for each school. For multivariate analysis, normalized on 
schools to a z-score metric [M = 0, SD = 1]. 

School SD, 9th Grade Mathematics Grades. A school aggregate; the within-school standard 
deviation of 9th grade mathematics grades [average of the school standard deviations = 
.96, standard deviation of the school SDs = .19]. For multivariate analysis, normalized on 
schools to a z-score metric [M = 0, SD = 1]. 



Weight 

School-Level Weight. For all school-level analyses, descriptive and multivariate, we employ 

S1SCWT3, which is the Qian-Frankel weight [normalized to have a mean of 1 on our 184 
schools]. 



Appendix D: A brief description of typical hierarchical linear models used in the study of 
curriculum structure and mathematics achievement 

Within-School Models 



A simple form of Hierarchical Linear Model (HLM) used in this study consists of two equations, 
a within- and a between-school model. Some of the parameters estimated in the within-school 
model become outcomes to be explained in between-school equations. Our within-school model 
investigates the mathematics achievement at the end of high school of student i in school j, Yy, as 
a function of a student's k social and academic background characteristics, Xjjk, and random error, 
Ojj. The X-variables discussed here are socioeconomic status (SES), gender, minority status, and 
students’ grades in mathematics in the first year of high school, and the highest level of 
mathematics course the student has taken over the four years of high school. A typical HLM 
within-school equation is as follows: 

MATHACHjj = Pjo + PjiSESy + p^jEEMALE^ + PjjMINORITYy + 

Pj4MTHGDESij + PjjMATHCRSij + Rij- 

Pjk regression coefficients are structural relations occurring within school j that 
indicate how achievement within each school is distributed across the measured student 
characteristics. In the HLM model that we investigated here, we were particularly interested in 
two P parameters: 

Pjo = the average 12th-grade math achievement score for students in school j (adjusted for 
academic and social background), and 

Pji “ the relationship between SES and achievement among students in school), which 
we refer to as the SES/achievement slope. 

In this model we initially estimated all distributional effects (the p parameters) in our HLM 
analyses. However, we were not interested in modeling the additional parameters as functions of 
the school organization or curricular structure. The limited sample size (and resulting degrees of 
freedom) within schools constrains our ability to estimate random par am eters, but it does not 
limit the number of fixed variables that may be included as statistical controls. Consequently, 
between-school variability in the other within-school controls (gender, minority status, 9th-grade 
grades, and coursetaking) is typically constrained to 0 (i.e., they do not vary randomly between 
schools). Thus, a final within-school model might be: 



Pij = Pjo + PjiSESij + P2FEMALE + . . . + p^MATHCRSij + Cy. 



Between-School Models 



In the second set of equations, we model the random-effect |3 parameters, adjusted for student 
characteristics, as functions of school-level organizational and structural characteristics (W- 
variables). We estimate a single between-school model for each outcome, estimating the effects 
of contextual features on the two outcomes (Po and Pi). A typical between-school model is as 
follows: 



Pkj - Yko + ykiWij + YkjWjj + ... + YkpWpj + Uy. 

The parameters of interest here are the effects associated with aspects of the school’s curriculum 
structure (W variables), represented by the yp,, (gamma) coefficients. In this instance, we might be 
interested in modeling between-school differences in adjusted math achievement as a function of 
two curriculum structure (W) variables: the average number of low-level math courses taken 
(LOWCRDTS) and whether the school offers calculus (CALCULUS). Although this model is 
considerably simplified from the full HLM models in this paper, a typical HLM between-school 
model might be: 

AVMTHACH (pjo) = Yoo + y.oLOWCRDTSj + yjoCALCULUSj + Ujo. 

In this model, we interpret the gamma coefficients as school effects on average achievement as 
having following meaning: 

7oo = grand mean for adjusted 12-grade mathematics achievement; 

y,o = the effect of the average low-level courses taken in each school on average 
mathematics achievement; 

720 = the effect of calculus availability in each school on average math achievement. 

Similarly, we could model P„ the SES/achievement slope — a direct measure of the social 
distribution of achievement in a school — as a function of factors describing school curriculum 
structure. 

SES SLOPE (pji) = 7o, + 7i,LOWCRDTSj + yjiCALCULUSj + Uj,. 

Gamma coefficients, which quantify school effects on this measure of social equity within 
schools, have the following interpretation: 

7 o, = grand mean for the relationship between SES and achievement (typically this is a 
positive relationship); 



Yii the effect of the average low-level courses taken in each school on the 

SES/achievement slope (a positive effect increases inequity; a negative effect means 
more social equity); and 

Y21 ~ fhe effect of calculus availability in each school on the SES/achievement slope 
(similar meaning for positive and negative effects). 

Because error terms in HLM within-school (e^) and between-school models (U^j) are complex, 
conventional linear model techmques (e.g., analysis of variance or ordinary squares regression) 
are inadequate for the estimation procedure. However, developments in statistical theory and 
computation, available through the HLM software, make such estimation possible with 
maximum likelihood procedures (see Bryk and Raudenbush, 1992 , for details). Briefly, the total 
variance in each outcome is partitioned into two components: parameter and error variance. Only 
effects on the parameter variance are estimated in HLM. This is an important distinction, as it is 
only variability in the structural parameters, Var(pj,J, which can be explained by factors 
describing curriculum structure. Efforts to estimate school effects with general linear model 
methods have typically resulted in their being systematically underestimated for this reason. 
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